25/07/16 - this post is kinda old. There’s some useful discussion in the comments as to what the more up-to-date approach is; essentially, you should look into ExpectedConditions.

If you’ve ever used AngularJS, you’ll know that it has a pretty cool built-in testing setup - Karma for unit testing, and Protractor for end-to-end testing. I love Protractor, but recently ran into issues when trying to use it for a non-Angular project. Here’s how to get it working.

If you’ve not encountered end-to-end testing before, you’re in for a treat. Unlike unit testing, which is designed to validate very small components, an end-to-end test can be used to simulate how a user will actually interact with your website. We can make it go to a page, click a link, check the right page gets loaded, give it some input, check it displays the correct output - whatever you want, really. Used in conjunction with other forms of testing, it gives us a really useful look at how our site’s working, and is great for helping us catch any sneaky regressions.

To get started (skip this if you just want to get to the React stuff), we need NPM. Do this:

$ npm install --save-dev protractor
$ ./node_modules/protractor/bin/webdriver-manager update

This will:

  • Download and install Protractor locally. Feel free to leave off the --save-dev flag if you don’t want the dependency added to your package.json.
  • Fetch, or update, your local Selenium and ChromeDriver instances.

Selenium is a software framework that lets us automate web browsers, and ChromeDriver lets it talk to Google Chrome specifically. I tend to use Chrome for these - you can, of course, use Firefox, or even PhantomJS, though the latter does suffer from some issues with Protractor.
Unfortunately Selenium and ChromeDriver aren’t included in the actual NPM install, but the included script gets them set up very quickly.

Next, let’s write our config file, protractor.conf.js, for Protractor. Mine’s pretty basic, and looks like this:

exports.config = {
    specs: ['*.js'],
    capabilities: {
        browserName: 'chrome'
    },
    baseUrl: 'http://localhost:5000',
    framework: 'jasmine'
};

What do these bits do? Well…

  • specs: This tells Protractor how to find our test files. In my case, they’re all JavaScript files in the same directory as the config file.
  • capabilities: There are lots of options can go in here, but for a simple setup the only one you really need is browserName, which tells Protractor which browser to run your tests in. It’ll look to see if there’s already a Selenium server running, and if not, will spin one up for Chrome.
  • baseUrl: This tells Protractor where to point our automated browser to.
  • framework: This tells Protractor which test framework to use. Jasmine provides a really nice syntax, check it out.

There’s a reference config file that has an explanation for every single one of the configuration options available. Have a look if you want something a little more specific.

Next up, let’s create a few really basic tests for our login flow…

describe('app login flow', function() {

    var loginUrl, homeUrl, name;

    it('sets up initial variables', function() {
        // Can be considered to be beforeAll, which Protractor lacks.
        browser.get('/login');
        loginUrl = browser.getCurrentUrl();
        browser.get('/');
        homeUrl = browser.getCurrentUrl();
    });

    it('registers a user and redirects to home', function() {
        browser.get('/register');
        name = 'user' + Math.floor(Math.random() * 100000);
        $('#email').sendKeys(name + '@test.com');
        $('#email2').sendKeys(name + '@test.com');
        $('#username').sendKeys(name);
        $('#firstName').sendKeys('Test');
        $('#lastName').sendKeys('User');
        $('#passwd1').sendKeys('Secret123');
        $('#passwd2').sendKeys('Secret123');
        $('button').click();
        expect(browser.getCurrentUrl()).toBe(homeUrl);
    });

    it('logs in correctly', function() {
        browser.get('/login');
        $('#username').sendKeys(name);
        $('#passwd').sendKeys('Secret123');
        $('button').click();
        expect(browser.getCurrentUrl()).toBe(homeUrl);
    });
});

Hopefully it should be easy enough to understand what these tests are doing.

First, we set up some initial variables. Protractor gives us a beforeEach function we can use, but not a beforeAll. Thankfully, the it functions are executed in-order, so we can use one without any assertions for some setup.

We point the browser to the registration page, and use the built-in CSSish selectors to grab the elements we want and send text to them. We then click the button, and check that we’ve been redirected to the home page. The browser object and $ function are provided by Protractor - you don’t need to require them in or anything.

We then do something similar with the login page, checking that it redirects, on a correct login, to the home page.

We use a randomly generated user each time - as Protractor tests the front end only, we don’t have any control over getting the database to clean after tests, unless we wire it up somehow. You probably don’t want to run these tests on a site in production - though I’ve had some success using Protractor tests to measure load.

Now all we have to do is run the tests. You can simply point the Protractor executable at your config file, like so:

$ ./node_modules/protractor/bin/protractor protractor.conf.js

If you’re like me and you like using a task runner such as Gulp, you can write a Gulp task to do the same thing:

{protractor} = require 'gulp-protractor'

gulp.task 'test-e2e', ->
  gulp.src e2eDir + '/*.js'
    .pipe protractor {
      configFile: e2eDir + '/protractor.conf.js'
    }
    .on 'error', (e) ->
      throw e

I wrote the above in CoffeeScript, just for a change. Note that you’ll need the gulp-protractor package, install it in the usual way.

Now that we’re all set, we should be able to run our tests, watch as the browser pops up and flies through the tasks we’ve given it, and hopefully everything’s passing. Cool. This should all work nicely for an AngularJS app, but for other frameworks a little more legwork is required.

The bit about React

Well, I say React, but really this applies to anything that isn’t Angular. I’m using React at the minute and I like it quite a lot, but I digress.

One problem we immediately run into trying to use Protractor with something that isn’t Angular is that when we run the tests, it just sits there. And sits there. And then times out, with an error that looks like this:

Error while running testForAngular: asynchronous script timeout: result was not received in 11 seconds

What Protractor tries to do is find the actual Angular instance that is running on the page it’s pointed at. In our case, that instance simply isn’t there, so it doesn’t know what to do. The reason for this is that Protractor is built with Angular in mind, and has loads of extra goodies for working with Angular apps. We shouldn’t be forced to use these if they’re not relevant to us, right?

Right. Thankfully, there’s a very easy fix. Simply add the following to your protractor.conf.js:

onPrepare: function() {
    browser.ignoreSynchronization = true;
}

This just tells our browser, “don’t wait to sync up with Angular before doing stuff” - and it works! Problem solved, then?

Well, not quite. Have a look at this test again:

it('logs in correctly', function() {
    browser.get('/login');
    $('#username').sendKeys(name);
    $('#passwd').sendKeys('Secret123');
    $('button').click();
    expect(browser.getCurrentUrl()).toBe(homeUrl);
});

This test fills out our login form, clicks the ‘submit’ button, and then checks the URL of the resulting page to make sure it’s redirected properly. On an Angular site with synchronization on, it works perfectly. However, if we’re using React or something else, and sending our login stuff as an Ajax request to the server, we’ll notice it fails - the test will immediately jump in and compare the URLs, before we’ve even had time to send the request. This means browser.getCurrentUrl() returns the URL of the login page. What’s going on?

When we run this test against Angular, Protractor hooks into Angular’s $http backend, and waits until its work is done before continuing. In other frameworks we have no such thing, and so Protractor has no idea what to wait for, so it doesn’t wait at all. This means it has no idea what’s going on with your Ajax calls.

It would be nice if there was an elegant way around this, but I’ve been unable to find one. The best solution I’ve come up with is simply simulating the time for the request to be run, and getting Protractor to wait for a given period of time. Thankfully, Protractor includes a call to do just this - browser.sleep(), which returns a promise. Our above test therefore becomes:

it('logs in correctly', function() {
    browser.get('/login');
    $('#username').sendKeys(name);
    $('#passwd').sendKeys('Secret123');
    $('button').click();
    browser.sleep(1000).then(function() {
        expect(browser.getCurrentUrl()).toBe(homeUrl);
    });
});

This isn’t pretty, but it gets our tests passing again. I would estimate that even one second of sleep is probably much longer than necessary if your tests and backend are being run on the same machine. For requests that take longer to process, you may want to play with the values. For testing against a server running remotely, you might want to give it a fairly long sleep time - better a slow test than one with false negatives, I think.

If there’s a nicer way to solve this little problem, I’d love to hear about it. It might be possible to tweak Protractor to plug into another JavaScript library, such as SuperAgent, but that’s more work than I was prepared to put in for now.

There are end-to-end testing frameworks other than Protractor, which are a bit more framework-agnostic:

However, Protractor’s ease of setup and configuration, as well as nice integration with my existing Node setup, makes it something I like using.

I hope this write up comes in handy for someone. Any questions, suggestions, improvements or short poems, please do leave a comment below.

Header photo via John Fischer on Flickr.