Engineering Blog

In the first post in this series, I explained what the engineering team calls a convention test, and why convention tests are important. This time around, I’m going to explain what a typical convention test might look like.

So far, all of our convention tests are all intended to prevent classes of problems. Part of the idea is that by writing a single test, we can avoid writing dozens or hundreds of additional tests. That’s not the important part. The important part is that we don’t need to remember to write that type of test any time we’re implementing similar code.

The convention tests we’ve written so far each use static analysis to examine a particular set of classes and fail the build if those classes contain violations. With that in mind, here’s what a convention test might look like:

List violations = newArrayList();
for(Class potentialViolator : ProductionClasses.findAll()) {
    // add any violations to the list
}
if(violations.size() > 0) {
    String violationString = Joiner.on(“\n”).join(violations);
    String fullError = String.format(“There were %s classes that ”
        + “engage in aberrant behavior XYZ. This is bad because ”
        + “it has the potential to cause apocalypse ABC. See defect ”
        + “DE12345 for more details. Here are the problems:\n%s”,
        violations.size(), violationString);
}

Here are a few key notes about this pattern:

• This is an answer to the obvious first question: ProductionClasses is a class we wrote that exposes a single method: findAll(). That method iterates all of the classes in our production classpath. Typically, that’s the type of thing we’re concerned with when writing these tests. Some tests iterate other problem sets, but this is a good example.
• We are working with all classes in this case. Other times, we work with all JSPs in our source path or all Type Definitions in a sample Rally subscription. The idea for a convention test is that you are testing all potential instances of a problem at the same time.
• We’re not failing with a cryptic error. The error message clearly explains what the developer did wrong and what to do about it, along with the repercussions.
• The technical explanation of the problem is valuable, but the context available outside the codebase is equally important. The error message includes the ID of a defect in Rally for us to refer back to.
• We’re not failing on the first violation. By telling the developer exactly how many violations there are and where they live, it immediately gives some idea of how large the problem is. This is important when dealing with a widespread problem.

What constitutes a violation? That’s a question for another post.