Engineering Blog

This one’s going to be short and sweet, with no sample code. Recently, I’ve been blogging about writing effective integration tests.

If you’re going to make the time investment (both development time and execution time) in writing integration tests, you need to make them work as much like your production environment as possible. Exercise as much of the same stack as you can.

If you’re writing to a database, you should be using the same connection pooling and transaction management mechanism as production. If you’re using an inversion of control container, you should be using the same one as production and loading as many of the same components as possible.

Why? If you don’t, your integration tests are not really proving anything. If you’re committing to a database, but not the same way you will in production, that leaves room for data inconsistencies. If you’re only loading part of your IoC container, that leaves room for unexpected wiring of dependencies.

Make your integration tests like production. It’ll save you from bugs later.

In my last post I promised to share some tips that can help developers write effective integration tests.

Committing to a database is expensive. In many standard SQL databases, it’s pretty much the most expensive thing you can do. By comparison, querying is relatively inexpensive. If possible, you should structure your integration tests in such a way that you assert as much as possible for each commit that’s made.

Consider this test. Rally supports the notion of a hierarchy of projects, structured into a tree. Individual users can have access to any node(s) within the tree and should only see data from those projects. This test creates users and projects and verifies our permission model.

public void whenOnParentScopedDownOnlyIncludeChildProjectsWhereUserHasAccess(PersistenceContext context) {
  Project parent = createProject();
  Project editorAccessChild = createProject(parent);
  Project viewerAccessChild = createProject(editorAccessChild);
  Project noAccessChild = createProject(parent);

  User user = createUserAsProjectEditor(parent);
  makeProjectEditor(user, editorAccessChild);
  makeProjectViewer(user, viewerAccessChild);
  context.commitAndReset();

  ProjectOidsInScope scope = scopeTo(parent).as(user).includeChildren();

  assertThat(scope.get(), containsOidsOf(parent, editorAccessChild, viewerAccessChild));
}

public void whenOnChildScopedUpOnlyIncludeParentProjectsWhereUserHasAccess(PersistenceContext context) {
  Project noAccessGrandParent = createProject();
  Project viewerParent = createProject(noAccessGrandParent);
  Project editorProject = createProject(viewerParent);

  User user = createUserAsProjectEditor(editorProject);
  makeProjectViewer(user, viewerParent);
  context.commitAndReset();

  ProjectOidsInScope scope = scopeTo(editorProject).as(user).includeParents();

  assertThat(scope.get(), containsOidsOf(editorProject, viewerParent));
}

These are good tests. With a little understanding of the Rally domain, they’re highly readable and they test the right things.

So what’s the problem? We’re committing twice when once will do. While the impact on any individual test is fairly minimal, the impact of unnecessary commits is compounded when spread across thousands of tests. Even within this test class, these aren’t the only two tests. There are ten.

What if we wrote the test like this?

private Project editorProject;
private Project editorAccessChild;
private Project viewerAccessChild;
private Project noAccessChild;
private User user;
private Project noAccessGrandParent;
private Project viewerParent;

@BeforeMethod
protected void setupData(PersistenceContext context) {
  if(dataCreated) return;
  dataCreated = true;
  noAccessGrandParent = createProject();
  viewerParent = createProject(noAccessGrandParent);
  editorProject = createProject(viewerParent);
  editorAccessChild = createProject(editorProject);
  viewerAccessChild = createProject(editorAccessChild);
  noAccessChild = createProject(editorProject);

  user = createUserAsProjectEditor(editorProject);

  makeProjectViewer(user, viewerParent);
  makeProjectEditor(user, editorAccessChild);
  makeProjectViewer(user, viewerAccessChild);

  context.commitAndReset();
}

public void whenOnParentScopedDownOnlyIncludeChildProjectsWhereUserHasAccess(PersistenceContext context) {
  ProjectOidsInScope scope = scopeTo(editorProject).as(user).includeChildren();

  assertThat(scope.get(), containsOidsOf(editorProject, editorAccessChild, viewerAccessChild));
}

public void whenOnChildScopedUpOnlyIncludeParentProjectsWhereUserHasAccess(PersistenceContext context) {
  ProjectOidsInScope scope = scopeTo(editorProject).as(user).includeParents();

  assertThat(scope.get(), containsOidsOf(editorProject, viewerParent));
}

We’ve moved the data creation into a setup method that runs (and commits) only once. We’ve sacrificed a small amount of readability within each test because the data setup isn’t directly adjacent to the actions and assertions. We have, however, gained some consistency across tests. Once we know the data structure we’re working with, we don’t need to mentally parse each test separately to figure out what shape the data is in before reading the rest of the test.

Integration tests are expensive, so achieving maximum bang for buck is important. If you can verify multiple behaviors while only doing the expensive part of the test once, DO IT!

The Strategy Pattern is a new favorite design pattern of mine. Probably because it has made my life substantially easier on a bunch of occasions. You can use it for situations when you’d like to group together a family of algorithms and interchangeably drop them into other code units (e.g. objects) at run-time. These strategies are readily reusable, composable and mixed in to your code.

Traditional implementations of the Strategy Pattern implement it with function pointers or first-class functions. These are stored away and later selected according to some given set of requirements. For example, say I write some awesome code for binding Subscription domain objects with incoming HTTP request data in a flexible manner. I can label my binding service as a strategy (suitable) for Subscription objects and they’ll automatically pick it up for binding.

In the Rally code base we implemented the Strategy Pattern using Java annotations. Our framework uses reflection to get classes annotated @StrategyFor and stores them in a Map. The association between consumers and strategies is based on type. Since types are hierarchical in Java we can write strategies for more than just a single class: we can annotate as a @StrategyFor some parent class and cover an entire sub-hierarchy.

Here’s a code example. Let’s say toward the end of some transaction we want all Artifacts to be processed and have their custom attributes set. So:

@StrategyFor(Artifact.class)
public class ArtifactPostProcessor implements DomainObjectPostProcessor {

    @Override
    public void process(DomainObject domainObject, Map arguments) {
        Artifact artifact = (Artifact)domainObject;
        artifact.setAllCustomAttributeValues();
    }
}

Ok, so now you’re thinking, if a family of objects needs a given behavior why not just create some interface and have them all implement it? Sometimes that is too tightly coupled: changing the interface means having to change every single implementing object.

The Strategy Pattern is an alternative that encourages building objects by composition over inheritance. Using composition means you can change the description of the superclass (what would be the Strategy class) without breaking the subsclasses.

Neat, huh?