Engineering Blog

In my last post, I promised another post about how we’re capturing metrics on our build health. This post is that post – or at least the first of those posts.

When we decided to “protect the build light,” making the data visible inside of Rally The Application (RTA) made sense. We all use RTA all day, every day. Using the app to make our data publicly visible seemed like the best way to get it in front of as many sets of eyes as possible, as often as possible.

For a while now, RTA has had support for tracking Build and Changeset data. We use Jenkins as our continuous integration server, and we’ve written a Jenkins plugin to publish build data from Jenkins into RTA. The plugin records the type, start time, duration, and status of every build in Jenkis. All of that data can be read and written to using our public, versioned Web Services API.

The Rally App SDK makes writing custom screens within RTA pretty easy, as long as you’re familiar with javascript. I’ve used the App SDK from time to time in the past, but it was mostly while tinkering with existing apps. Putting together a board that displays our build metrics seemed like a great way to learn more about the framework.

First, when working with the App SDK, you need to include the SDK script file:

<script type="text/javascript" src="/apps/1.27/sdk.js"></script>

Once you have the script included, at the heart of the App SDK is the DataSource. It abstracts away a lot of the boilerplate code associated with talking to the server. Instantiating a data source is easy. This snippet will create one that track whatever workspace and project you currently have selected in Rally. If you want to track a specific workspace/project, replace the __WORKSPACE_OID__ and __PROJECT_OID__ with specific ids.

// create a datasource scoped to current workspace and project, not scoped up, scoped down
var rallyDataSource = new rally.sdk.data.RallyDataSource(
  '__WORKSPACE_OID__', '__PROJECT_OID__', 'false', 'true');

In order to work with the data source, we need to build queries. Our query is built from an array of references to build definitions within RTA. We OR-together a series of expressions so that builds from all of the build definitions are included. We then fetch only the attributes we’re interested in, to limit network traffic.

var buildDefs = [
  '/builddefinition/1', // build 1
  '/builddefinition/2' // build 2
];

var queries = [], firstRef = buildDefs[0];
var query = new rally.sdk.util.Query("BuildDefinition = " + firstRef);
for(var i = 1, len = buildDefs.length; i < len; i++) {
  query = query.or("BuildDefinition = " + buildDefs[i]);
}
queries.push({key: 'builds', type: "build", order: "Start desc", query: query,
  fetch: "Number,BuildDefinition,Duration,Start,Status" });
rallyDataSource.findAll(queries, me._renderBuilds);

A few notes here: _renderBuilds is a function we’ll explore in a minute. It’s a callback that gets invoked asynchronously once the ajax findAll completes. Next, the App SDK query class does a nice job of encapsulating the ridiculous balanced parenthesis syntax used by queries in our web services stack. Finally, the json object we push onto the queries array has some interesting keys and values. Type refers to the object type within our web services stack. Order specifies the sort order for the returned items. Fetch specifies the attributes that should be returned. Key specifies the property name in the json results under which the query should be returned.

me._renderBuilds = function(items) {
  var data = [].concat(items.builds).sort(function(build1, build2) {
    // sort by end time, ascending
    return endTime(build1) - endTime(build2);
  });
  me._calculateMetrics(data);
  if(showDetails) { me._renderRawBuildData(data); }
};

As promised, here’s the _renderBuilds function. Early on, we access the builds property of the items parameter. Items is the result of the ajax call we made above. It’s a JSON object. The builds property is the data we requested above, using the key we specified. The code in the if(showDetails) block is just there for debugging if something seems amiss. The important parts of this method are the call to _calculateMetrics and the sort algorithm. Sorting by start time when querying for the data is not sufficient for reasons I’ll go into in another post. The call to endTime leverages the Duration and Start properties of the build to come up with an accurate end time.

me._calculateMetrics = function(data) {
  var tableDiv = setUpSomeDiv();

  var metricKeys = ["Metric","Data"];
  var metricsConfig = { 'columnKeys': cols, 'columnHeaders': cols, 'width': TABLE_WIDTH, columnWidths: ["35%", "65%"]};
  var metricsTable = new rally.sdk.ui.Table(metricsConfig);

  dojo.forEach(data, function(build) {
    dojo.forEach(metrics, function(metric) {
      metric.sample(build);
    });
  });
  dojo.forEach(metrics, function(metric) {
    metricsTable.addRow({
      Metric: metric.name,
      Data: metric.calculate()
    });
  });

  metricsTable.display(tableDiv);
};

Okay, there’s a lot going on here.

First, we create a div in which we display our results. I’m intentionally glossing over that part – it’s the least important part here.

A key point is that in several places in the function, we reference an array named metrics. That array contains objects that should conform to a simple interface. Each metric should have a name property and two functions: sample and calculate. Sample should accept a single build and store enough information about that build for its calculate method to display something meaningful.

Next, we establish an array to represent the columns in our metrics table and use that to construct the config object for the table. Because both column headers are just one word, we use the same array for the column key set. Many lines later, when we call addRow on the table, the names in the column key array are used to pull properties from the JSON objects we pass to addRow.

After that, we iterate the entries in the data array that was passed to the function. Inside that loop, we also iterate the metrics array to call sample on each build row. Finally, we iterate the metrics array and create a row for each metric. Ultimately the app builds a nifty table that looks something like this:

There are some interesting metrics there. Four of them are pretty easy to calculate – the other two are significantly harder. I’ll explain more about the individual metrics some other time.

What did we miss?