Guest blogger John Firebaugh is an avid Ruby and Javascript developer, an OpenStreetMap mapper, and a contributor to the OpenStreetMap.org web site. He joined us for a sprint this week in the MapBox office in Washington DC where, among other things, he focused on testing.

It is extremely important that the iD editor is stable and tested: when deployed, it will be used to modify the singular OpenStreetMap database. To ensure that iD’s growth is accompanied by stability and reliability, we spent time this week refining the choice of tools we use for testing, improving test coverage, and setting up a continuous integration system. You can now check out iD’s testing status at any time on Travis-CI.

Automated testing helps us improve iD’s architecture and technical design (through test-driven development) and reduces the chances of inadvertently breaking existing code as we add new features and fix bugs. In this post I’ll outline our test infrastructure and explain how to test your iD instance when developing.

Testing Tools

iD runs in a browser, as client-side Javascript that relies on a DOM. We’re using a combination of testing tools that allow us to test both interactively in a number of browsers, and automatically, whenever a change is submitted to the repository.

  • Mocha provides the framework for defining tests, grouping them logically into suites, and running them. We use it in the style of behavior-driven development, whose main constructs are describe, to group a set of related test specifications, and it, to specify a facet of behavior. We use a top-level describe to name the module under test, second-level describes for each of its methods, and as many its as are necessary to fully specify each method’s behavior. For a good example, have a look at the tests for iD.Entity.

  • Chai provides the language for test assertions, with which we declare exactly how we expect the modules and functions that make up iD to behave. For instance, with chai’s assertions we can say that we expect(iD.Node({tags: {foo: 'bar'}}).tags).to.eql({foo: 'bar'}).

  • Sinon provides test spies, stubs, and mocks. Some parts of iD lend themselves to being tested in isolation, some parts require that their direct collaborators be mocked or stubbed, and some are best integration tested, with only the connection to the OSM API stubbed out. In all of these scenarios, we can take advantage of the test doubles provided by Sinon (even in isolation testing, spies are handy for verifying that callbacks are triggered at appropriate times). And with sinon-chai, they are all nicely integrated with chai’s assertion language.

All Together

Here’s how we confirm that the DeleteNode action actually removes a node from iD’s graph database:

// The group of tests
describe("iD.actions.DeleteNode", function () {
    // A specific desired behavior
    it("removes the node from the graph", function () {
        var node   = iD.Node(),
            action = iD.actions.DeleteNode(node),
            graph  = action(iD.Graph([node]));
        // The assertion: if graph.entity(node.id) is defined,
        // a test failure is reported
        expect(graph.entity(node.id)).to.be.undefined;
    });
});

iD tests run within the browser, so we can easily open different browsers and confirm there are no platform-specific bugs. In-browser testing also makes it possible to use Chrome’s Web Developer tools to pinpoint the cause of a failure.

Testing Your Changes

How do you run the tests locally, after making a change to the code? It’s easy: just open your repository’s copy of test/index.html in a browser. This file includes each of the individual script files that comprise iD, so rerunning the tests after a change is as simple as refreshing the page.

A companion file, index_packaged.html, tests the concatenated and minified iD.min.js package. It catches some problems that index.html doesn’t, but it requires running make after any change.

We encourage you to run the tests before submitting a pull request, and to add tests for any changes or additions you make.

Continuous Integration

To ensure that the tests pass after every commit, we rely on Travis CI, an open source, sponsor-supported project that provides free continuous-integration testing for projects on GitHub. When a change is pushed to GitHub, Travis is notified via post-receive hook and runs and reports tests, emailing the maintainers on test failures.

The tests run in-browser, so how do they run automatically? We use phantomjs, a headless WebKit runner, which integrates with Mocha via mocha-phantomjs providing a console-driven test runner and reporter. Luckily, travis-ci has phantomjs installed by default, so it runs perfectly on their servers.

Let’s Build (and Test)

The second ingredient for our testing is people - try it out and report issues and bugs! Combining continuous integration testing with reporting and management of bugs will make iD a rock-solid editor for OpenStreetMap.



Devlogging work on the OpenStreetMap project by the MapBox team.

Much of this work is currently focused on improvements to OpenStreetMap funded by the Knight Foundation

Follow our work here on this blog or subscribe to our Twitter feed. You can subscribe to this blog’s feed or follow us at