So, you have a Ruby on Rails project you've been testing with Capybara and capybara-webkit and you need to upgrade to Headless Chrome.

Well, you're in the right place as here I'll show exactly how you can achieve that. But first, let me provide some context about why it's important to make the change, and why Chrome is the perfect candidate for it.

Since 2017, Google Chrome has been shipping with a headless environment, allowing us to emulate user interactions without the overhead of having a GUI. They also partnered up with Selenium, a browser automation tool to release ChromeDriver. It provides a standard interface to control Chrome, so it'll play nice with most tools and languages that want to use it.

Now, before all this was an option, capybara-webkit, the driver for the QtWebkit browser, was a great match for the job. However, I wouldn't say the same now. The tests fail intermittently, forcing retries on the CI, and the browser it relies on (QtWebkit) has been deprecated.

All things considered, with Chrome you've got a modern browser, a driver for it, and a field-tested tool to automate your tests (Selenium). All that without needing to handle pesky Qt version dependencies. Awesome, right?

Alright, now that you're all up to speed, I'll walk you through setting up ChromeDriver and Selenium, while providing a fix for some of the most common issues that may come up.

Setting up the environment

Making the switch requires Chrome, of course, and a couple of dependencies to make sure everything's neatly integrated with Capybara.

This part's fairly straightforward, it shouldn't be much of hassle for you.

Installing Chrome

First things first, you need to install Chrome's latest stable version
In Linux, you can do so as such:

wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add - 
sudo sh -c 'echo "deb https://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list'
sudo apt-get update
sudo apt-get install google-chrome-stable

Installing Dependencies

Next up, you'll need two more tools:

• ChromeDriver: Chrome's implementation of WebDrivers interface for remote control;

• Selenium: needed to implement the automation and testing tools that you'll use with Capybara.

The gem webdrivers helps with the installation of ChromeDriver, automatically downloading, installing and keeping the driver up-to-date.

Then, you should add both to the project in your Gemfile, as shown below:

gem 'webdrivers', '~> 3.7', '>= 3.7.2'
gem 'capybara-selenium', '~> 0.0.6'

Don't forget to bundle install afterwards.

Setting up the drivers

Now, you just need to register the drivers, and configure them in spec_helper.rb:

Capybara.register_driver :chrome do |app|
  Capybara::Selenium::Driver.new(app, browser: :chrome)
end

Capybara.register_driver :headless_chrome do |app|
  capabilities = Selenium::WebDriver::Remote::Capabilities.chrome(
    chromeOptions: {
      args: %w[headless enable-features=NetworkService,NetworkServiceInProcess]
    }
  )

  Capybara::Selenium::Driver.new app,
    browser: :chrome,
    desired_capabilities: capabilities
end

Capybara.default_driver = :headless_chrome
Capybara.javascript_driver = :headless_chrome

This sets the default driver to :headless_chrome. If you'd like to watch the tests execute, just change it to :chrome in the last two lines.

You may also notice the enable-features tag in chrome's options, this is a temporary fix because of an issue in Chrome 74 in which cookies get randomly cleared during execution, which might cause Chrome to freeze.

According to Chromium’s bug tracker, this will be fixed in version v75.

If you're running the project in Docker, you may also need to add 'no-sandbox' to Chrome's options:

args: %w[headless no-sandbox enable-features=NetworkService,NetworkServiceInProcess]

Cleaning things up

Now that that's taken care of, you can go ahead and remove capybara-webkit from the Gemfile, as well as any import or configuration you might have left (look for Capybara::Webkit).

Issues that might come up

For some projects, the tests may already be running smoothly after these steps, but for others that may not be the case. With a new browser and tools running the tests, new features and new problems also come up.

Capybara-webkit had a couple of useful but non-standard methods, and Selenium does not support all the methods Capybara has to offer. So this creates quite a gap, and any test that was using unsupported methods has to be patched.

• Element#trigger is not supported by Selenium, although in most cases it should not be used, since it allows actions that the user will never be able to do.

Nevertheless, an easy workaround, if you’re triggering a click, would be to send the return keystroke to the element:

find('#yourElement').send_keys(:return)

Or to click the element through javascript:

execute_script("document.querySelector('#yourElement').click();")

If you're dealing with another sort of event, you can use jQuery like so:

execute_script("$('#yourElement').trigger('event')")

• resize_window is renamed to resize_window_to in Selenium.

• Reading JavaScript logs is a bit different. In the driver configuration you'll have to change the capabilities to something such as:

capabilities = Selenium::WebDriver::Remote::Capabilities.chrome(
    chromeOptions: {
      args: %w[headless enable-features=NetworkService,NetworkServiceInProcess]
    }
    loggingPrefs: {
      browser: "ALL",
      client: "ALL",
      driver: "ALL",
      server: "ALL"
    }

And then to read the logs, you can simply:

page.driver.browser.manage.logs.get(:browser)

You can read more about Chrome's capabilities and options here.

• Inspecting and setting the requests' headers is not supported in Selenium by default. This means that page.driver.header, page.response_headers and page.status_code aren't available.

Fixing this last point is somewhat of a challenge, but GitLab's solution is a great workaround. When faced with the same problem while porting their browser from PhantomJS to Chrome, they implemented a Middleware to intercept the requests' headers (more about it here).

To implement this solution, I simply included these files

lib/testing:

• request_blocker_middleware.rb;

• request_inspector_middleware.rb.

spec/support/helpers:

• inspect_requests.rb;

• wait_for_requests.rb.

The namespaces have to be changed to match your project, and the 'concurrent-ruby' gem imported, as it is required in the middleware:

gem 'concurrent-ruby', '~>1.0', '>=1.0.5'

Remember to also import the files in spec_helper.rb:

Dir[Rails.root.join("spec/support/helpers/*.rb")].each { |f| require f }

And with all that setup, all you have to do to get the header's details is the following:

requests = InspectRequests::inspect_requests do
    visit some_path
end

puts requests.first.url
puts requests.first.status_code
puts requests.first.request_headers
puts requests.first.response_headers

A couple more tips

Now that everything is up and running, let me share a few more tips to take the most of Headless Chrome & ChromeDriver and avoid some common issues.

Animations can be a hassle, especially scroll animations

Capybara clicks on elements in the following way:

1. Find DOM element;
2. Calculate element coordinates;
3. Click on said coordinates.

If the page is, for example, scrolling when the element is meant to be clicked, the coordinates might get outdated between step 2 and 3, meaning that the click will fall in the wrong place.

One possible solution for this problem is to wait for the animations to end, in this case I waited for the jQuery animation scrolling the body to stop:

Timeout.timeout(Capybara.default_max_wait_time) do
    sleep(0.1) until page.evaluate_script('$("body:animated").length==0')
end

Another option would be to disable jQuery animations in testing altogether, like this:

page.execute_script("$.fx.off = false")

It's worth noting that disabling the animations can also improve the tests' performance.

If this fix doesn't work for you, and you really want to cancel all animations, check out this great article put together by the folks at Doctolib.

Fixed elements stealing your clicks

Capybara only clicks on elements if they are visible, so if you have a navbar or a popup obscuring an element, you might get an error like this:

Element is not clickable at point (100, 200). Another element would receive the click: ... (Selenium::WebDriver::Error::UnknownError)

To deal with this, you can close all popups on the page, and scroll down to the element before clicking it.

A simple method, implementing this idea, would be:

def scroll_to_and_click(query)
    el = find(query)

    options = "{ block: 'center', inline: 'nearest' }"
    script = "document.querySelector('#{query}').scrollIntoView(#{options})"

    page.execute_script(script)
    el.click
    
    el
end

Conclusion

Chrome's headless mode and ChromeDriver that comes with it have been strongly adopted for testing and automation, especially since QtWebkit was deprecated, and, with it, projects that were based on it, such as PhantomJS and capybara-webkit.

Even the maintainer of PhantomJS, the once popular headless browser has deprecated his project in favor of ChromeDriver. And thoughtbot, the creators of capybara-webkit, are starting to play around with ChromeDriver as well.

While capybara-webkit did the job for quite some time, the change to a more modern alternative (Chrome's headless mode) will make tests more reliable and stable. All of this with the additional advantage of using the same browser engine as most users, which makes the tests actions much more similar to what a real-life user interaction would look like.

Happy testing folks!

Found this article useful? You might like these ones too!

• Node.js and Ruby on Rails compared
• Ruby on Rails: paginate stateful tabs with pagy
• Node.js: Strapi and Express Admin reviewed