In the first post in this series, I talked about what we call convention tests. In the second post, a general outline for a convention test was given. This time, let’s look at a more specific example.

There’s a fair bit of Java/Servlet/JSP-specific stuff in this post, but much of it is applicable regardless of your language of choice.

For a long time, we’ve been working on tightening up our release cycles. Over time, six releases a year has turned into 40+, ideally releasing every week. Sometimes, even a one week release cycle is too long, so we’re working toward “rolling releases.” In a rolling release, new code is rolled out to our VMs one at a time, without ever impacting the user experience. Often, users don’t know anything changed until a new feature is available or a bug is fixed in the middle of their session.

The actual process of performing a rolling release is what recently led us to start precompiling our JSPs. Once we started releasing intraday, we discovered a concurrency bug in the particular JSP compiler we use. If multiple users hit the same JSP at precisely the same time for the first time after a server restart, it would hork up (that’s a techinical term) that JSP until the server was restarted again. This problem manifested itself most often in the highest traffic pages in Rally, because those were the most likely to be hit concurrently.

Here’s an example of the dialog in the office around that time:
(let’s go play foosball until we figure this out)

It turns out that in the servlet container we’re using, precompiled JSPs conform to a VERY different lifecycle when compared with dynamically compiled JSPs. They are pooled across requests rather than instantiated per-request. This can lead to very bad behavior if any state is stored in tags used in the JSPs.

Something like this can cause problems if ${object} contains data that has the potential to be stale:

<prefix:sometag someattribute=”${object}” />

If the data sometag stores for someattribute is not reset during the next request on the same thread, it will show as stale. A common reason the data might not reset is if the backing value for someattribute is stored in a field on a custom tag. The servlet spec says that all tags should implement a release method that resets all stored state to default values when called. If the release method is not implemented, or if it fails to reset a field, the stale data problem can manifest.


In the second article in this series, the code sample had an intentionally mysterious comment that said

// add any violations to the list

In that post, I suggested that we’d cover the topic of what constitutes a violation in another post. I covered it two paragraphs ago. In this case a violation is not resetting a field on a tag class when release is called.

In pseudocode, the body of our violation checking loop looks something like this. For an example of what the outside of the loop might look like, see the second post in this series:

if potentialViolator is a tag {
    find all fields defined in rally class hierarchy
    create an instance of potentialViolator
    for each field found in step 3 {
        capture the initial value
        set a new value that is different from initial value
    call release
    for each field not reset to initial value {
    add a violation for each field that was not reset

Line by line notes:

1. potentialViolator is a class, as detailed in the second post in this series.
2. Classes defined in a library not owned by Rally are assumed to be safe. This could be a hole in our test, but it hasn’t proven problematic so for. It also eliminates the noise when trying to work the list and eliminate problems.
3. We are relying on a default (zero-argument) constructor, because the JSP library does as well. Java makes it trivial to instantiate a class if it exposes a default constructor.
4. Self explanatory
5. We are reflectively examining the fields and capturing the initial value in a map declared around the same time as line 3.
6. This is where the magic happens. When dealing with primitive fields, we assign random values to numeric and string fields, or the inverse of the initial value for boolean fields. For fields that are non-primitive object references, the new value is set to a mock generated by mockito. In mockito, calling .equals() on a mock will evaluate to false when compared to null or to a real instance of the class in question. If you are working in another language, a good understanding of that language’s notion of object equality and the behavior of the mocking framework you are using will help you here. The key idea is that you are setting the field to some value other than its default.
7. }
8. We call the release method reflectively and throw with a descriptive error if we can’t call it.
9. To check whether a value was reset, we compare the value after calling reset to the value before calling it. The value after calling release should be the same as the initial value captured on line 5. This is once again done reflectively. A key point is that by doing it reflectively, we eliminate any data transformation that may take place in the getter.
10. The violation text in this case is a concatenation of class name and field name.
11. }

I’m happy to share code that works for us upon request. The exact code for this test has some specificity to our codebase. I excluded it so that the language and tool specifics wouldn’t add noise this post. I don’t feel like it’s anything confidential – it’s just noise. You need to find the implementation of each step that works for you, but I’m happy to help.

Happy testing!

Request a Call

Looking for support?

Send Us Your Feedback

Provide us with some information about yourself and we'll be in touch soon. * Required Field