Engineering Blog

One of the most common sources of tension between product owners and developers is when product owners are surprised at how high an estimate for a story might be. Usually this tension is easy to resolve by reiterating that the product owners really have no concept of how much something should cost. However, there is one scenario I see over and over again: when a product owner protests the estimate of a story because it seems, to the PO, like it’s simply re-using an aspect of the system somewhere else.

“I already paid for that!” he or she might say, quite correctly. A simple example recently for us here was our column picker. Our grids allow you to select which columns you wish to see on any given grid. Later on, we created the ability to have grids inside of panels on user dashboards. Our product owner wanted to put the column picker on the settings pane of a grid panel. To him, this was simply reusing the column picker that had already been created for an earlier story. He was quite shocked when our estimate for the effort was just as high as it was the first time.

His frustration is completely understandable. How is this possible? Why isn’t it easy to just drop that component into another place and have it work? After all, if we were good engineers, wouldn’t we have written that component to be reusable?

Actually, no. It’s because we were good engineers that it wasn’t written in that way. This is one of the hardest things for product owners to understand, and I hope to explain it in this post.

Evolutionary Design

Once upon a time, development teams worked in specific phases. First they would design all of the code they were going to write; boxes would be drawn on whiteboards, sequences captured in documents, and so on. Once the design was finished, the developers would go off and write the code for it. Luckily, our industry has realized that this is a problematic way to write software.

It’s simply not likely that the team will discover all of the quirks of the design during the design phase. It’s even more likely that requirements and features will change during the long span of time this process takes, forcing the developers to throw away work and go “back to the drawing board” on the design. Splitting the design phase from the coding phase is a recipe for disaster.

Instead, it’s generally preferable to practice evolutionary design; allow the design to grow organically as the code is written. This, combined with testing and refactoring, seems to be the best known way to ensure that the quality of the code remains high as a product grows.

Doing this makes it easier to deal with changes that effect design, and it prevents the team from wasting time going “back to the drawing board” in the event a problem is discovered.

YAGNI

One principle to follow as you code is the YAGNI principle. YAGNI means “You Ain’t Gonna Need It” and essentially it means that developers should discourage themselves from adding additional code to support something that is not, at the time, required. Developers have a strong desire to add code here or there while editing code, and YAGNI helps us remember to resist the urge to add superfluous code or functionality.

Following this principle helps eliminate waste, and helps keep developers focused on adding value that product owners want. But why?

If I’m editing Module A to add Functionality X, why not just add Functionality Y while I’m there? It may delay the delivering of Functionality X in the short-term, but later when it’s decided that Functionality Y must be supported as well, it will be dirt cheap, right?

The trick to understanding why YAGNI helps is realizing that there are thousands upon thousands of potential Module As in a system. The truth is, Functionality Y may simply never be required. And even if it is, Functionality Z may not be. Adding that functionality would be a waste of time and resources if it’s never used.

YAGNI Applied to Design

The same principle behind YAGNI can be extended to aspects of high-level design. When we originally designed the Column Picker component, it was not designed to be dropped into any part of the system. This was intentional: if we had spent the time to make the Column Picker completely generic and reusable, but then had never needed to drop it into another part of the system, the additional effort required to make it super-generic would have been a wasted effort. This is true for any aspect of the system being designed: we can spend the extra time and effort to make something extremely generic, but if that generality is never needed, that time is waste. Many times developers will refer to something that’s overly generic as “over-engineered.”

This means that the description of the component can often be misleading. We never built “a column picker” – we built “a column picker for a grid page”. The column picker was written in a way that assumed it was part of a grid page. Again, it could have been written more generically, but doing so would have potentially created waste (this does not mean that it’s alright to design code that violates principles of good design. Code can be well-designed but still not fully generic).

So when it came time to put the component in a completely different context than the one for which it was designed, it was still a lot of work: almost as much as was originally required to make the first pass. Though some time was saved simply because it was a known and well-understood problem for which we would leverage newly-developed knowledge, it still effectively required writing a whole new component: one that was much more generic and utilized what it could of the previous increment.

Essentially, to avoid having developers waste time, components should be “generic on demand”. Once a component needs to be reused, then it should be refactored into something extremely generic, but not before. The end result is that a product owner will have to pay for a component twice before it’s cheap.

This may seem counterintuitive to product owners, but it’s important to understand. Because the team approaches coding in a way that causes the second time to be nearly as expensive as the first, the team is able to deliver everything else quickly.

The first time, it is costly because the developers have to solve unknowns and create something new from scratch.

The second time, it is costly because it has to be molded into something generic and reusable.

The third time, however, is cheap.

During the month and a half leading up to the holiday break, a group of five developers (including myself) volunteered to form a temporary Scrum.  The goal of this Scrum was to build Rally’s next generation of load-testing tool.  The need for a better tool surfaced when our current tool failed to forewarn us of a problem that eventually caused a brief production outage.

For the first couple of weeks, we looked at commercial tools focusing mainly on IBM’s Rational Perfomance Tester.  We presented our findings to the Engineering team, and the engineers decided that we should spend a few weeks trying to build our own load-testing tool from scratch.  The tool would use Jmeter, which was the tool that our current load tests were written in.  However, in the new tool all of the logic that was previously in Jmeter would be written in Java, and Jmeter’s sole purpose would be to serve as the “engine” to run the newly-written Java code.

After we had built a working prototype, we again gave a brief presentation to the Engineering team.  The decision was made to go with the framework that we had built internally.  There were many reasons for this decision, but the biggest deal-maker was that the developers felt it would be easier to integrate the home-grown tool into our development process because it was written in Java and hence we could write the tests in IntelliJ just as we do for the Rally application.  Additionally, we could use IntelliJ’s debugger to debug the tests which was a huge plus.

The original prototype that was presented to the Engineering team was not written using Test Driven Design; in fact there were no tests written at all.  We said something like, “Oh it’ll just be a quick prototype to see if we can do it.  If we decide to continue down this path instead of RPT, then we’ll scrap it and re-write it with TDD.”

Well, as you probably guessed, two weeks later after the decision had been made to continue down the path of building our own load-testing framework, we didn’t scrap it and re-write it.  No, we couldn’t do that – too much work had already been put in to building the prototype to throw that all away, right?

So we decided that from then on we would use TDD for new code and we’d write tests for the areas of our already-written code that we felt needed tests.  Using this method, we took our load-testing tool from prototype to finished product.  During this time, I witnessed first-hand the benefits of writing tests before writing code, as well as the notion that writing tests drives out design flaws.  I’d like to share a few of my observations:

1. TDD Code Looks Like It Was Written By Wall-E

During my first five months at Rally, I had noticed that in a lot of our code, there seemed to be an overuse of delegation.  Many methods consisted of one line that would simply delegate to another class, which would in turn delegate to yet another class and so on.  This was very different from anything I’d seen at previous jobs and at school where code was more procedural, more “human-like”, and read more like a book (you could many lines of code in sequence without jumping around much).  The code in the Rally application looked like it was written by a robot.

However, while I was doing TDD on our new load-testing framework, I realized a benefit of “robot-generated” code: it makes writing tests much easier.  In our prototype, we had the notion of Gesture classes that had Gesticulation annotations on them.  Gesticulations contained certain metadata about the Gesture.  Every Gesture extended the abstract class AbstractGesture class which had accessor methods to the Gesticulation data.

When we added an enhancement where we needed to do some complex processing on the Gesticulation within the AbstractGesture class, I decided to use TDD.  However, I couldn’t instantiate an instance of AbstractGesture because it was an abstract class.  Therefore, the need to write a test before writing code drove out the need for a separate class whose sole responsibility was to handle all of the logic dealing with Gesticulations.  AbstractGesture delegated all questions about it’s Gesticulation to this new class which was written using TDD.

2.  Changes Are Really Easy To Implement in Well-designed, Well-tested Code (a.k.a. code written with TDD)

We spent a day or so refactoring a single class that we had written during the prototype stage; we essentially ended up rewriting the entire class from scratch.  The need for this refactoring was driven out by the fact that it was impossible to write tests for this class after all the code was written.  Additionally, we needed to make an enhancement to this class that was also very difficult to do given the way that the code had originally been written.  After we spent the day re-writing with TDD, the enhancement change was literally a one-line change!  I’d like to think that had we written the class originally with TDD, we would have been able to make the one-line change without spending a whole day refactoring.

3.  Test Driven Refactoring Can Lead to the Discovery of New Lands

We had a section of code that was written during the prototype stage that essentially looped through a list of Gestures, plucking out those that met certain criteria based on data in their Gesticulations.  The code for this was essentially a for loop with an if statement inside of it that decided whether the Gesture should be “plucked”.  As I began to write tests for this code, testing the logic inside of the if statement and the for loop seemed way more painful that it should have been.

So I asked a few of our engineers and they suggested looking at the Google Collections library.  This lead me to the filter(Iterable<T> unfiltered, Predicate<? super T> predicate) method in the Iterables class.  Using this method, we were able to refactor the code to be much simpler and easier to test because all of the logic dealing with deciding whether or not the Gesture needed to be plucked was now coded in a newly created Predicate class (which, of course had tests around it).  Additionally, the code needed to loop through the list was essentially gone (in that it was implemented in the Google Collections code).

4. Writing Tests Can Be Fun

Although I had written many unit tests in the past, I had never enjoyed writing them and I don’t think I really understood the full benefits of having them around.  However, as I was doing TDD and Test Driven Refactoring while working on our new load-testing system I saw for the first time how writing tests before writing code can lead to a more simple, extendable design.  Additionally, I truly enjoyed writing the tests because they were driving out design flaws and showing me how to write better Object-Oriented code.  It was so easy a robot could’ve done it .

How do you tell the difference between a web application developer, and the Average Joe who’s not actually a web app developer but did stay at a Holiday Inn Express last night?

Well, if you ask them how they feel about Internet Explorer and they respond with an expletive-filled tirade of hate and disgust, good chance they are actually a web application developer. Sure this is probably an over-exaggeration, but the Average Joe out there would probably never think of the need for testing an application in all of the major browsers (for Rally, this is IE, Firefox, Safari, and Chrome). They’d probably think something like, “If it works in Firefox, why wouldn’t it work in all browsers?”, or “All of the web applications that I use look nearly identical in all browsers so all browsers must work the same”. They couldn’t be more wrong.

The engineers at Rally have spent many hours ensuring that our users can use our application in whatever browser they choose, assuming they choose one of our supported browsers. (The same can probably be said about the engineers at nearly all of the other popular web applications in use today.) The majority of our time spent making our app cross-browser compatible is spent on making it work in Internet Explorer. It’s for good reason too: approximately 50% of our users access the Rally application with Internet Explorer, compared to 41% for Firefox. Of the users that use our app with Internet Explorer, 60% of them use Internet Explorer version 6.

I’d like to discuss a few of the interesting IE6-specific problems we’ve solved recently:

1. ExtJS’s Ext.layout.BorderLayout doesn’t render the splitter element in Internet Explorer 6

We created the Widget Catalog window on our new Dashboard Beta page by extending Ext.Window in the following manner:

RALLY.ui.dashboard.WidgetCatalogWindow = Ext.extend(Ext.Window, {
 title : "Widget Catalog",
 width : '70%',
 height : 400,
 layout : 'border',
 ...
});

As we were putting the CSS styles on this window, we noticed that the dark-blue vertical splitter element that separated the two panels of the window didn’t appear to be rendering in IE6. However, it was present and colored properly in every other browser, as shown below:

We tried to put a border on the splitter element and the border still did not appear in IE6. However, when you moused over where the splitter should be, the mouse pointer changed to a drag-type pointer, and you could still drag and drop the splitter as you would expect so it seemed like the element was there, but IE6 wasn’t coloring it for some reason.

After poking around for awhile, we stumbled upon a web page that lead us to believe we should mess around with the CSS “filter” property. This did the trick:

.ext-strict .ext-ie6 .widget-catalog-window .x-layout-split, .widget-catalog-window .x-layout-split {
 background: #0d345f none !important;
 filter: none;
}

2. The multiple class bug in Internet Explorer 6

If you have access to more than one Workspace in our application, there is a “Workspace Picker” available to you in the upper-left hand corner of the application:

Notice the border surrounding the words “Workspace 1″. This border should show up if BOTH of the following conditions are met: the user has access to more than one workspace AND the user’s mouse is positioned over the workspace picker element. If both these conditions are not met, the border should not appear on mouse over.

We thought that the following CSS would adequately meet our needs:

#main #header div.simple-picker.selectable.sp-over {
 border: 1px solid #3f84a4;
}

“sp-over” is the class that gets added to the workspace picker on mouse over, and “selectable” is the class added when the users has access to more than one workspace. Our problem was that, in Internet Explorer 6 only, the border was rendered whenever the user’s mouse was over the workspace picker, regardless of whether they actually had access to more than one workspace. This was caused by what Paul Irish deems “The Multiple Class Bug” in his blog post here.

Essentially IE6’s CSS parser was interpreting the “div.simple-picker.selectable.sp-over” selector as simply “div.sp-over”. Therefore, on mouse over, the “sp-over” style was being applied to the workspace picker element regardless of whether it had the “selectable” class and hence a border was shown.

Once we had figured out the problem, a workaround was not difficult to implement. We decided to create a new CSS class just for IE6 that would be applied to the workspace picker element when BOTH conditions for the border to be shown were met. We added this in the onRender(…) method of the WorkspacePicker, which extends Ext.Component:

RALLY.ui.picker.WorkspacePicker = Ext.extend(Ext.Component, {
 onRender : function(ct, pos) {
   if (this.el && this._isSelectable()) {
   this.el.addClass('selectable');
     if(Ext.isIE6){
       this.el.addClassOnOver("sp-over-selectable-combined-class-for-ie6");
     }
   }
 }
 ...
});

Additionally, we needed to modify the CSS to use this new class:

#main.ext-ie6 #header div.sp-over-selectable-combined-class-for-ie6,
#main #header div.simple-picker.selectable.sp-over {
 border: 1px solid #3f84a4;
}

However, this wasn’t quite good enough for IE6. A border will still be applied to the workspace picker when it is not selectable because the second selector (#main #header div.selectable.sp-over) will still match when the mouse is over the workspace picker (remember, IE6 reads “div.simple-picker.selectable.sp-over” as simply “div.sp-over”). So, we added the following CSS to make the border color the same color as the background (so that, to the user, it would appear as if there was no border) if the sp-over-selectable-combined-class-for-ie6 was not present on the workspace picker element:

#main.ext-ie6 #header div.simple-picker {
 border: 1px solid #085478;
}

Interestingly, this newly-added selector is more specific than the “#main #header div.simple-picker.selectable.sp-over” selector in IE6’s eyes, but only because of the browser’s crappy CSS support. If you calculate the specificity of the original selector according to standard rules, you would come up with a specificity of 2, 3, 1 (number of ID, class, and element selectors). However, remember that IE6 sees “div.simple-picker.selectable.sp-over” as simply “div.sp-over”, so in IE6’s eyes the specificity of this selector is actually 2, 1, 1. The specificity of our newly added selector (#main.ext-ie6 #header div.simple-picker) is 2, 2, 1. Therefore, our newly-added selector will be applied by IE6.

Finally, please note that all of the CSS selectors added to fix this problem use the handy ext-ie6 class, which is conveniently added by Ext only in the case when the application is accessed using IE6. Additionally the added JavaScript code is wrapped by the Ext.isIE6 check that only returns true when the application is accessed using IE6. Both of these mechanisms made this problem much easier to solve.

These are just a few of the many interesting IE6-specific problems we’ve solved recently. Solving browser-specific problems is a big part of web application development but I feel like I’ve I only started to scrape the surface in this area, because, truth be told, five months ago I was just an Average Joe who stayed at a Holiday Inn Express . Ah, how much I’ve learned.

“To wrap or not to wrap, that is the question.”

Most of the time when we think of word-wrapping in a browser, we think of how to prevent it from happening at all. In fact, the use-case is so frequent, that you can even do it with this well-known bit of CSS:

white-space: nowrap;

What if we wanted to go the other way? What if we wanted to force text wrapping?

In normal circumstances, this is no problem. It’s done for us. All the modern user agents (browsers) out there are pretty good about wrapping text at appropriate places: whitespace, hyphens, and the like; basically, between words. This leads to text that is neatly left-justified and fits within its container when rendered (just take a look at this blog post for an example of this happening).

A problem occurs, however, when a line of text is both unbreakable (no spaces, hyphens, or other non-word characters) and too long to fit in its container. Take this example:

SuperLongFirstName_SuperLongLastName@Company.com

The sheer length of this email address simply blew out the 250 pixels allotted to hold it. To us, it’s just an email address. To the browser, however, it’s a single (but very long) word. Since the browser won’t break up individual words, it won’t wrap this text for us.

Luckily, there two ways to fix this:

• Wait for CSS 3, and use word-wrap: break-word;
• Insert an invisible character that the browser will treat as a space, thus leveraging existing word-wrap behavior.

Given that CSS 3 is still a draft and not widely supported, it looks like option #2 is, for now, the way to go.

Luckily for us, most modern user agents support such an invisible character. It’s known as the “zero-width space.”

The HTML character entity is: &#8203;

If I insert this character prior to the ‘@’ sign in the e-mail address, like this:

<p class="email-box">
SuperLongFirstName_SuperLongLastName&#8203@Company.com
</p>

The text breaks neatly:

SuperLongFirstName_SuperLongLastName​@Company.com

The reason this works is fairly simple: the browser treats the zero-width space as a non-word character, and thus what was one (1) word is now two (2). Since the browser only wraps between words and now that you have two (2) words, it will happily wrap them to the next line for you.

In practice, however, I’ve found that it’s not always feasible to know at precisely which character one should break. If I did, I could simply just insert a <br /> and be done with it.

So, to get the most out of this workaround, I insert the zero-width space after every character that I need to wrap:

<p class="very-small-box">
A&#8203B&#8203C&#8203D&#8203
</p>

This way, for the string “ABCD”, the browser sees four (4) words instead of one (1) and wraps appropriately.

Sadly, nothing in this world is perfect. Remember, when you’re doing this, you’re inserting an invisible character — but a character nonetheless. If a user copies-and-pastes such an altered string, he’ll carry along those extra invisible bytes. This could look very interesting depending on the application he’s copying that data to.

I’ve tested and verified that this workaround works as expected in IE 7 and 8, Chrome 2 and 3, Firefox 2 and 3, and Safari 4. Unfortunately, this trick does not work in IE 6; instead of a zero-length space, you see the “unknown character” box. Oh well.

Has anyone else successfully solved this issue in another way? I’d love to hear from you.

Many developers think of unit tests as tests that test a single class. In fact, I myself once thought this way. If I wanted to write unit tests for a two-class system in which a class used another class, I’d write two unit tests. After all, if I created instances of both classes in my test, that wouldn’t really be a unit test, would it?

In recent years, I have revised my stance on this matter. This distinction between what is and is not a unit test is one I no longer draw in the same way.

Unit Tests need to be:

• Fast – Unit Tests should never run so slowly that they discourage developers from running them all the time.
• Focused – Unit Tests should focus on a single area so that they can isolate problems to that area when they fail.

Once upon a time, I thought ‘focused’ meant that it should be focused on a single class. Instead, it should focus on an area, but that area can include multiple classes.

(more…)

Mark Levison recently posted a challenge to Agile tool vendors, so I’ve written some responses below about our Definition of Done, TDD, acceptance testing, release frequency, and retrospectives.

Definition of Done
We separate out our definition of “Done” for individual stories and for the release as a whole.  Each release planning, we discuss it as a group and make updates, so this is current as of last Friday:

Story
All tasks complete
All tests running and passing
Manual walkthroughs complete on all fully supported browsers
Migrations and Web Services updates reviewed
Ops & On-Premise impact recorded
User gesture tracking added
Performance test results reviewed
All defects closed by testers
UX review complete
Accepted by Product Owners

Release
All stories accepted
All tests run & passing
Build passing
Performance tests run & accepted
Migration tested against trial & production
Outage time defined & communicated
Doc & external content complete
Agile Commons (our user community)
Announcements (pre & post)
Release notes
“What’s New?” screen updated
Interstitial light box

TDD and Unit Testing
We do TDD to different extents in different parts of the system.  TDD in Java has been relatively straightforward, but as a rich web application, we have fairly sophisticated JavaScript code that historically has been tricky to unit test.  In the past, we leaned heavily on our acceptance testing framework on the GUI side, but we’ve recently come up with a JavaScript unit testing approach that we’re happy with and that supports TDD.

Acceptance Testing
For the last few years, we’ve been writing tests in an internally built Ruby framework that drives Selenium for much of our automated functional acceptance testing.  This works, but the tests take a long time to run, and so we’re exploring other strategies.

We do support a lot of browsers for a relatively sophisticated client-side experience.  Firefox 3.x, IE 8.x, Safari 4.x, and Chrome 2.x are all fully supported.  For older browsers (Firefox 2.x, Safari 3.x, IE 7.x and 6.x) we provide functional support – Rally needs to work on these, but sometimes there are display issues with margins and the like.  We have one exploratory tester per scrum who does exploratory testing on each story on each of these different browsers.

We also do automated performance testing on each story to see the impact on overall performance, to be certain that new work doesn’t degrade our response times.  Because many of our customers are international, we include user gesture tracking so we can keep track of actual client-side performance on different browsers in different parts of the world so we know what performance users are experiencing after we release.  Our average server-side response time is under 200ms, and our average client-side response time (server time + network time + browser render time) is under 1 second for the top browsers.    Here’s some actual data for real customers this fall:

Finally, the product owners also do a manual walkthrough of each story as it is completed.  The team has already verified that the acceptance criteria are tested, but sometimes during the acceptance process I find missing acceptance criteria that turn into additional stories.

Release Frequency
We release about 40 times per year.  Patches with small features and fixes go out most Saturdays, and we have major feature releases 6 times per year so we can manage customer expectations and keep the rest of the company in sync.

Last Retrospective
I feel like revealing our retrospective contents in public might violate trust and a feeling of personal safety for our team.  One issue that came up recently was our test system and the extent to which it mirrors our production environment.  Each scrum holds a retrospective every other week, and we do a cross-scrum retrospective every 8 weeks.

But really, these things are table stakes – you’re probably going to get decent answers from most agile tool vendors.  There are other questions that seem to matter a lot more to our customers when they’re choosing a tool

- How well does the vendor match the level of engagement that you want?  If you want a deep, “partner” level of engagement, can you get it?  Or is the vendor going to dump a zip file in your lap and ignore your emails?
- How smooth are your releases?  Will we have to spend a lot of time adjusting how we work each time you change things?  Or do you put effort into building features that are easy for users to adopt and non-disruptive?
- How’s your performance internationally?  If we have teams all over the world, what actual client-side performance do users of your product experience in Mumbai?  In Singapore?  In Chicago?

As a user experience professional, my job is to provide development with highly-usable designs that best match the goals of our customers.  But in the fast-paced world of agile software development and being the single UX resource for several teams, I am not able to put every design through the same rigorous User-Centered Design (UCD) process in which I was trained.  Instead, I must analyze the risk of each upcoming feature and determine my approach.  As part of this risk analysis, I consider many of the following questions:

• Is this an additive feature or a change to an existing feature?
• How disruptive will it be to the user’s current process?
• How often will our users use it?
• How many of our users will be impacted by it?
• How critical is the functionality to our users?
• How complex is the design and is there potential for many different alternative designs?
• How well do we understand the goal of the feature and how it will be used?
• Have we already done something similar to this that we can adapt for this purpose?
• Is there another application that has a good design that we can adapt for this purpose?
• How difficult is it to change the design if we get it wrong? (from a user acceptance perspective)
• How difficult is it to change the design if we get it wrong? (from a development effort perspective)

If I determine a feature to be high-risk based on these criteria, I will dedicate a greater amount of my time to bring it through the UCD process.  For a low risk feature, my effort will be greatly reduced.

For example, a few releases ago, we were working on a feature to allow subscription administrators to override user’s individual session timeout settings in order to match their internal security policy.  This feature was primarily additive and the goal very straightforward.  The number of users that would interact with the feature would be a very tiny percentage of our user base.  The frequency that the administrator would change this setting would be low.  The design was simple and well-contained and the possible permutations on the design were relatively few.  Overall, this feature felt low risk.  As a result, I spent about an hour iterating on the design and did not perform any user validation.

In contrast, in the next release, we’ll be working on a very high risk feature – moving our Iteration Status page over to our new page framework.  It’s high risk for several reasons.  First, it’s the number one most used page in Rally and the usage of this page spans across our entire user base.  The possible design alternatives are myriad.  Even small interaction decisions would have a huge impact on user acceptance.  Finally, making significant changes after-the-fact would be costly from a development perspective and would result in significant re-learning from our users.

For this page, my work started about 2 releases out (about 3 months).  I iterated through many designs, conducted internal testing, had interactive prototypes built, tested these prototypes with customers and iterated again.  This work is still underway, with implementation beginning in about 4 weeks.  We will be further mitigating the risk involved by releasing this redesign as a Beta program, allowing us to gather and incorporate feedback before releasing the page to all users.

Many features fall somewhere in between these two extreme examples.  Based on the circumstances, I will draw on different UCD methods and techniques ranging from very lightweight to very heavyweight.  I might rely on feature request comments from our customer forum for one feature but for another feature I’ll schedule one-on-one calls with customers.  Sometimes I’ll perform usability tests with internal users and other times I’ll schedule formal usability tests with external customers.

Using design risk analysis to drive effort allocation is critical to helping me avoid delaying our development team.  With that said, because I come from a traditional UCD background, it can feel “wrong” to not fully validate each design with users.  Fortunately, the “release early, release often” mantra of agile is itself a risk mitigation technique.  In the case that a design is not well-received, we have an opportunity to listen to user feedback and release a better design in just a few short weeks.

This post is a continuation on a series of posts regarding our switch from YUI Test to JsTestDriver (jsTD).  In the first post,  Testing Javasciprt Is … I mean should be … Easy, I described some of the motivations for switching to jsTD as well as a few nice techniques for navigating and creating tests.  In this post, I’d like to talk about how we mapped our existing asserts from YUI-land to jsTD.

What We Had

YUI Test uses a TestNG-like syntax for defining the assert statements.  For example:

Assert.areEqual('foo', name, 'Should have been foo');

All assert statements are under the Assert namespace, and follow the (expected, actual, msg) format.

What We Want

JsTD uses a different syntax, more inline with the JUnit syntax:

assertTrue('Should have been foo', 'foo', name);

In this scheme, assert statements are in the global namespace and follow the (msg, expected, actual) format.

Our Challenge

When Rod and I sat down, our challenge was how to convert our existing Assert.assert*() statements into the assert*() statements that jsTD expects.  We have around 85 individual test files, and around 600 test methods.  This amounts to a significant amount of statements to fix during the conversion.

The Solution

Instead of manually converting each assert statement, we decided to create a mapping file to map from one syntax to the other.  This had a few benefits for us:

1. We didn’t have to touch any of our assert statements
2. It allowed us to use our preferred TestNG format (values and then an optional message), which is more consistent with our Java unit tests
3. It keeps the asserts in a top-level namespace of Assert

Thus our mapping file, test-setup.js, looks something like this:

var Assert = {};

(function() {

  var buildTwoArgList = function(actual, msg) {
      var args = [];
      if (msg) {
          args.push(msg);
      }
      args.push(actual);
      return args;
  };

  var buildThreeArgList = function(expected, actual, msg) {
      var args = [];
      if (msg) {
          args.push(msg);
      }
      args.push(expected);
      args.push(actual);

      return args;
  };

  // thanks Crockford
  var isArray = function(actual) {
    return (typeof actual.length === 'number' &&
          !(actual.propertyIsEnumerable('length')) &&
           typeof actual.splice === 'function');
  };

  // JsTestDriver supports the following:
  //
  //   expectAsserts(count)
  //   fail([msg])
  //   assertTrue([msg], actual)
  //   assertFalse([msg], actual)
  //   assertEquals([msg], expected, actual)
  //   assertSame([msg], expected, actual)
  //   assertNotSame([msg], expected, actual)
  //   assertNull([msg], actual)
  //   assertNotNull([msg], actual)

  Assert = {

      expectAsserts : function(count) {
          expectAsserts(count);
      },

      fail : function(msg) {
          fail(msg);
      },

      areEqual : function(expected, actual, msg) {
          assertEquals(buildThreeArgList(expected,
                                         actual, msg));
      },

      areNotEqual : function(expected, actual, msg) {
          try {
              assertEquals(buildThreeArgList(expected,
                                             actual, msg));
              fail(msg || 'Are not equal');
          } catch (e) { }
      },
      ....
      isArray : function(actual, msg) {
          assertTrue(buildTwoArgList(isArray(actual), msg));
      },

      isFalse : function(actual, msg) {
          assertFalse(buildTwoArgList(actual, msg));
      },

      isNotUndefined : function(actual, msg) {
          assertTrue(buildTwoArgList(typeof actual !==
                                     'undefined', msg));
      }
  };

})();

We then include the test-setup.js file in our jsTestDriver.conf config file between the source files and the test files.

With this in place, we were able to use all of our pre-existing test methods without touching any of the assert statements. It also allows us to define commonly used assert methods within the Assert namespace, and make them available for use in any test.

As a user experience designer at Rally, I’m thinking about 3 things at any given moment:  1) the work that is being developed right now in the current iteration, 2) the work that’s coming up next iteration and 3) the work that is coming up in the next 1-2 releases.

The Current Iteration

In order to avoid blocking the development team, my first priority is making myself available to answer questions and remove ambiguity from designs and acceptance criteria that may come up once implementation begins.  And for every user story that impacts the UI (most of which do), I have an explicit “UX Review” task.  To complete this task, I sit with the developer and review the work that was done.  For example, I just sat down with one of our developers this morning and walked through an error message dialogue that can appear after a bulk edit is made.  There were a couple of slight alignment and spacing issues that we identified and Katie quickly fixed these.  If the problem is significant and requires a large amount of development effort or rethinking the design, I work with the product owner to create a new user story to capture the additional work.  The new user story is then ranked accordingly – either to be done in the next iteration or sometime later.  Once the UX task is complete, our tester can perform a manual walk-through on all of our supported browsers.

The Next Iteration

My next priority is the the next iteration.  I frequently review the Backlog to identify what we will likely be working on next and as an extra measure I schedule a weekly meeting with the product owners to formally review the set of candidate user stories for the next iteration.  The assumption that the items are listed in rank order is critical to my work and ensures that I am creating detailed designs and acceptance criteria for the right set of work.  It would be wasteful for me to create detailed designs for work that may not be done for several weeks and it would much worse still for me to enter in to an iteration without a solid design for the team.

It’s a delicate dance, however.  Since story size is often dependent on the design and what will fit in an iteration depends on size there is often quite a bit of negotiation that goes on between developers, product owners and myself in preparation for the next iteration.  For example, if I create a design option A that is estimated at 10 points and design option B that is estimated at 5 points, we will weigh the options carefully.  Our developers are very accustomed to me asking them to take a look at a design and tell me how difficult it would be to implement.  Very often, they will suggest alternatives that would be technically easier and if an alternative is equally user-friendly, I’m happy to adapt the design.

On occasion the technical difficulty of a particular design does not come out until the formal process of iteration planning.  In these cases, the end result is usually compromised because the adapted design has not had the opportunity to be iterated on or validated.

The Big Picture

I have found that it’s imperative for me to know what is coming in the next 1 to 2 releases for two reasons:  1) to bring a holistic vision in to each design that I create and 2) to be able to plan ahead for larger, riskier features.  Fortunately, the product owners that I work with hold a Product Council meeting every 2 weeks with stakeholders and this allows me to gain insight in to what we will likely be working on in the next couple of releases.  When I’m not working on the current iteration and the next iteration, I spend my time thinking about these upcoming features and themes.  I identify which features will need customer research and validation and start to schedule these.  I explore high level design concepts and validate them.  This prep work ensures that I won’t be scrambling to get designs together when the work gets scheduled into an iteration and it also helps create a cohesive user experience over the long term.

We hear this all the time, “Slow down so you can speed up.” Recently the Engineering team here at Rally decided to take a slightly different approach.

Like most software projects that are several years old, our build system and directory structure were beginning to show their age and some cracks were starting to appear in the foundation. When we switched from Ant to Maven about 3 years ago we thought it would be a great idea to use Maven modules. Modules seem like a reasonable idea so we broke our code up into several. One for “server” code, those things that represent our domain model and all of their services. One for web application code, controllers, etc. And lastly, one module that was the guts of our web-app, CSS, JavaScript, JSP, etc. We also created a module for setting up the database and yet another one to configure the application server. Because of Maven’s convention over configuration getting all of these modules to work together and play nice required quite a bit of duct tape and bailing wire.

After a few years the team started to realize the way we broke down the modules doesn’t really make sense. Perhaps we want to break them up by functional area, or in some yet to be determined way. Regardless, we knew it would be more complex than what we currently have. Our solution was to simplify so we can make it more complex in the future. We did this in two ways.

First we collapsed all code into a single Maven module. We are now completely aligned with Maven’s conventions, so the build is no longer a mysterious black box, all developers can enhance it without worrying about breaking it. Additionally, I always know where code is, and the tests run a little faster without jumping between modules. When we’re ready we can start pulling things apart again.

Next we decided the application server configuration and deployment should be separate concerns. Our Operations team kicks ass, so why not let them be directly responsible for how to deploy things without having to edit the code base? Ops gets to do what they do best, and our code base loses a few thousand lines of build configuration. Seems like a good thing to all of us.

What did all of this get us? One thing that can’t be underestimated is the increased level of developer happiness with a clean and simple build and directory structure. Our tests run faster and there is much less directory changing.

Anyone else ever done something similar? Let us know.