Engineering Blog

As of late, we have been doing a large amount of front-end development in Javascript.  Most of this work is to support some awesome new features in Rally – like the dashboard and grid framework.  We are finally starting to eat our own dog food by consuming the same web services we have provided our customers for years.  As part of this development, we are ensuring that all new Javascript code is unit tested.

Due to our Javascript testing framework selection, it is difficult and (slightly) time consuming to continuously run the unit tests after each change.  Using YUI Test, we have to switch context out of our IDE to a browser to run the test.  This context switch can easily kill any flow you’ve reached while working on a story.   Additionally, we’ve found that Javascript tests seem to have a barrier for entry that is higher than that of Java unit tests.  This barrier can be enough to persuade even the best of us to skirt around creating a new unit test for a small change.

Given these challenges, and as part of our evolving build process, we’ve decided to focus on making Javascript testing easier, faster, and more tightly integrated into the development cycle.  To accomplish this, we’ve decided to switch testing framework.  We’ve also made strides at integrating these tests into the IDE and maven build process.  To help you avoid some of the pitfalls and challenges we’ve encountered, we want to share a few lessons learned.  If you take these and apply them, testing your Javascript will be … I mean should be … easy (or at least a bit easier).

Easy Navigation From Source to Test

This is key.  The more time you spend trying to remember what the name of the test is and where it’s located, the less time you’ll have to actually write meaningful tests.  If you use IntelliJ, I’m sure you find shift-command-T incredibly useful to switch between a the Java class and its unit tests.  But it doesn’t work for Javascript.  No need to worry, AltN8 comes to the rescue.  You can install this IntelliJ plug-in, setup up some file mappings and key-bindings, and you’re in business.  For example, if your Javascript tests follow a convention of appending Test to the name of the source file, you could setup the following rules:

1. ^(.*?)\.js$    ==>  $1Test.js

2. ^(.*?)Test.js$   ==>  $1.js

I also like to change the key-mapping from alt-8 to shift-command-J so that it more closely matches the key-binding for Java unit testing.  With this setup, from your source file, throw down a shift-command-J key sequence and viola,  you’re in the corresponding unit test (if there is more than one match, you can choose from a list).

Create Test Templates

Some test frameworks, like YUI, require a fair amount of setup around the actual test code.  In YUI test, you actually create a HTML file, include the relevant YUI sources, and finally define your tests.  You can type this each time you create a new test – if you feel like practicing your typing for a few hours.  You could open up an existing test and copy & paste to create a new test – but it’s almost inevitable this will lead to copy & paste errors and stale setup code.  A final option is to use your IDE’s templating mechanism.  For IntelliJ, you could easily create a live template to make this task much less intensive.  While the YUI test setup is very verbose, let’s look at a template for another testing framework, JsTestDriver:

TestCase('$TEST_CASE_NAME$', {
  test$NAME$ : function() {
    $BODY$
  }$END$
});

Now when you need to create a new test, just create a blank file, command-J, and type the name of your live template – for the above I use TestCase.  You can then tab through the replacement fields and put in your values.  Beautiful.

Similarly, you could create a live template for adding a new test method to a pre-existing TestCase:

  test$NAME$ : function() {
    $BODY$
  }$END$

With this template, you can easily add a new test method from within the TestCases prototype,

Select a Good Testing Framework

There are many good Javascript testing frameworks out there.  It’s up to you to pick the one that makes the most sense for you project.  We have done a lot of testing with the YUI Test framework, but have recently decided to make the move to JsTestDriver.  There are good and bad points to both.  For us, the speed and simplicity of the tests in JsTestDriver made it the clear winner.

In YUI Test, tests are defined in HTML files and run manually from the browser.  We’ve added some niceties around this, such as a JSP runner that would iterate over all HTML test files in a given directory, run the test in an iframe, and report failed tests when they happen.  However, there is a lot of overhead with loading up a separate HTML file for each TestCase or TestSuite.  Additionally, the barrier to entry for creating a new test was large.  In fact, too large.  There was a lot of HTML and Javascript/CSS includes that you needed to setup, instantiate a logging class, and define the tests.  This makes the tests long and difficult to read.

We wanted to remove all the HTML wrapping the tests, and just have the tests.  Removing the actual tests from all the surrounding cruft seemed to make them much more readable and maintainable.  This is exactly what JsTestDriver is setup to do.  In this framework, you run a server that is able to serve out all your source and test files.  You then point at least one browser at the server, and run the tests by issuing a command in the terminal.  Each browser registered with the server loads up the source files, the test files, runs the test cases, and reports the results.  This provides a distributed test environment with the ability to run your tests on a myriad of different browser/OS combos.  It integrates with continuous build systems, and runs wickedly fast.

But, there are some issues:

1. How do we map between Asserts.*() and assert*()?
2. How do we override or modify globally namespaced objects?  (the source files are no longer reloaded before each test, thus removing the built-in sandboxing of YUI Test)
3. How do we build up DOM dependencies needed to run the test?
4. How can we automate the start server, attach browsers, run tests cycle?
5. What about asynchronous testing – like YUI Tests’ wait() method?

In some follow-up posts, I’ll talk about each issue in more detail and how we have (or are trying) to solve it.  Stay tuned!