Automated Cross-Browser Tests with JavaScript

Cross-browser tests are extremely important for production applications if you want to ensure your users have a smooth experience across multiple browsers and devices. In this post, I'll explain how I automate these cross-browser tests by running my Selenium end-to-end tests across multiple browsers via BrowserStack.

If there was a lot of jargon in the previous sentence that you don't understand then the next few of sections are for you.

Selenium

Selenium is a popular and extremely useful tool that allows you to write code that will fire up a web page in a given browser, interact with the page using code, and examine the page for any errors.

End-to-end Tests

I won't go into detail on this but, in short, end-to-end tests are where you test that the whole application works as expected from start to finish of the user flow. For example, for an email app you would check:

  • The user can login
  • The user can open emails
  • When an unread email is opened, it is marked as "read"
  • The user can reply to, forward, delete, and compose any email
  • The user can log out

BrowserStack

BrowserStack is an awesome tool that takes the pain away from setting up Selenium on your local machine. Through its online tools, you can access all of the common browsers and different versions instantly, without installing them on your own machine. It also offers an automated service, which we'll be using here. This allows you to use Selenium on the BrowserStack cloud, testing across multiple browsers and versions in parallel.

Start Testing

I'm going to assume that you have an application up and running locally that you want to test and good knowledge of ES6. Your application doesn't actually have to be written in JavaScript to write your tests in JavaScript but, since you're here, it probably will be.

Setting Up BrowserStack

The first thing you'll need to do is get a BrowserStack account. Head over to the signup page and start a free trial. After the free trial, BrowserStack is a paid service but if you are developing open-source software you can get a free account simply by contacting them. Have a look at the pricing here.

Install the dependencies

Run the following command in your terminal:

yarn add selenium-webdriver browserstack-local chai dotenv wdio-mocha-framework webdriverio --dev  

If you're not using yarn and are using npm then try yarn, you won't regret it.

Config

Create a file named wdio.conf.js in the root of your project and fill it with the following code.

require('dotenv').config()  
var browserstack = require('browserstack-local');

exports.config = {  
  user:  process.env.BROWSERSTACK_USERNAME,
  key: process.env.BROWSERSTACK_KEY,

  updateJob: false,
  specs: [
    './src/**/e2e.test.js'
  ],
  exclude: [],

  commonCapabilities: {
    name: 'local_parallel_test',
    build: 'webdriver-browserstack',
    'browserstack.local': true
  },

  capabilities: [
    {
      browser: 'chrome',
      'browser_version': '60.0',
    },
    {
      browser: 'chrome',
      'browser_version': '55.0',
    },
    {
      browser: 'firefox',
      'browser_version': '55.0',
    },
    {
      browser: 'firefox',
      'browser_version': '46.0',
    },
    {
      browser: 'internet explorer',
      'browser_version': '11.0',
    },
    {
      'browser': 'Safari',
      'browser_version': '10.0',
    },
    {
      'browser': 'Edge',
      'browser_version': '15.0',
    },
    {
      'browser': 'Edge',
      'browser_version': '14.0',
    }
  ],

  logLevel: 'verbose',
  coloredLogs: true,
  screenshotPath: './errorShots/',
  baseUrl: '',
  waitforTimeout: 10000,
  connectionRetryTimeout: 90000,
  connectionRetryCount: 3,

  framework: 'mocha',
  mochaOpts: {
    ui: 'bdd'
  },

  // Code to start browserstack local before start of test
  onPrepare: function (config, capabilities) {
    console.log("Connecting local");
    return new Promise(function(resolve, reject){
      exports.bs_local = new browserstack.Local();
      exports.bs_local.start({'key': exports.config.key }, function(error) {
        if (error) return reject(error);
        console.log('Connected. Now testing...');

        resolve();
      });
    });
  },

  // Code to stop browserstack local after end of test
  onComplete: function (capabilties, specs) {
    exports.bs_local.stop(function() {});
  },

  // Add Chai
  before: function() {
    var chai = require('chai');
    global.expect = chai.expect;
    chai.Should();
  }
};

// Code to support common capabilities
exports.config.capabilities.forEach(function(caps){  
  for(var i in exports.config.commonCapabilities) caps[i] = caps[i] || exports.config.commonCapabilities[i];
})

This file will run any tests within src that are called e2e.test.js. All of the browsers that will be tested are specified in the capabilities array. You can change this to suit your needs by following this page.

You will also need to create a .env file in your root with your BrowserStack username and key. You can get these from the corresponding tab here once you are signed in. The file should look like this:

BROWSERSTACK_USERNAME=YOUR_USERNAME  
BROWSERSTACK_KEY=SOME_MAGIC_KEY  

Do NOT add this file to GitHub or any other public repository.

Adding the scripts

To your package.json file add this to "scripts":

"pretest:e2e": "echo 'ensure the local server is running on port 3000!'",
"test:e2e": "wdio ./wdio.conf.js",

Note, that the pretest:e2e script will just run before you run yarn run test:e2e in the terminal. You could spin up your local server (probably yarn start) here but I like to just add a reminder since it is likely I will have the local server running already.

Writing the Tests

The eagle-eyed among you may have noticed that I asked you to install WebDriver IO and Chai earlier, we'll be using these to write the tests.

I'm using the BDD (expect) style syntax for my tests but feel free to use whichever you prefer.

Somewhere in your src directory, add a file named e2e.test.js (probably at the root of src).

In this file you can create a simple test such as:

describe('My hello world app', () => {  
  it('says hello world when clicked', () => {
    // Assuming the port is 3000 this may be different for your project
    browser.url('http://localhost:3000');

    browser.click('#hello-world-button');
    expect(browser.isExisting('h1=Hello World!')).to.be.true;
  });
});

Obviously, this is just an example and you will need to write tests tailored to your app. For an idea of the kind of real world tests you can write have a look at one of the tests for a project of mine here.

Run the tests

Simply run yarn test:e2e (making sure your local server is running) in your terminal to start the tests. This will run all of your tests in the src directory in all of the browsers your specified in capabilities in the conf file, in parallel!

You will see an output to the terminal as the tests run but I find BrowserStack's web client more useful. If you navigate to https://www.browserstack.com/automate, you will see a very helpful rundown of all of the tests that have been run.

BrowserStack Automate Dashboard

By clicking on each test, you can see all of the individual tests that were run, what failed, and screenshots of where something failed - brilliant!

Done

That's it! BrowserStack Automate has made running multiple tests across many browsers in parallel so simple. Soon, I'll write a post on how you can integrate this all with Travis CI.

If you have any questions then don't hesitate to comment below.

Jacob Windsor

Likes creative computer-based stuff. Web, Science, Typography, Graphics... Trying to combine too many interests into something coherent.

Maastricht, Netherlands