RallyDev.com
Fri 10 Jul 2009
Stop Over-Committing! – Rethinking Utilization Limits
The Agile Blog is proud to present the first post from our newest contributor, Agile Coach Ben Carey
As part of our coaching at Rally, we usually recommend that new Agile teams use capacity as a countermeasure for not having a historical velocity. Considering individual and team capacity can help the team make an informed commitment for their iteration plan if they don’t have a historical velocity.
One general way to calculate the capacity can be seen below:
A team can use their individual capacities to indicate the maximum theoretical amount of work that each individual can complete in an iteration. Reviewing individual capacities against task estimates can help the individuals not over-commit and also help the team determine the amount of risk in committing to the stories that have been chosen for an iteration.
You can see your utilization percentage in Rally by looking at the Team Status page. As a general rule, I usually recommend that a team try not to exceed 70% of their capacity for each individual on the team.
There are many times that this recommendation is met with resistance because of the basic assumption that 100% utilization (total work divided by capacity) is the ideal state. I’m often asked to justify my general rule of 70% and asked why I would pick a number that is lower than the maximum.
Why I recommend capping individual capacity at 70%:
At a basic level, it seems to makes sense that we want everyone to run at full capacity. If the capacity exists, why shouldn’t we use it? Why would we ever want someone to do six tasks if they have the capability to do eight?
The problem with the 100% utilization mindset is that we have assumed that our outcomes of the iteration are fully deterministic.
When we consider the delivery of software, we need to acknowledge that a given set of inputs do not always result in a given set of outputs due to various degrees of variation in multiple aspects of the process (e.g. fire-fighting, estimation precision, defect introduction, management directives, interpersonal interaction, et cetera). In reality, software development activities are stochastic in nature.
If we look at other areas that are defined as being stochastic, we can build a mental model that can help us understand why 100% utilization isn’t the ultimate desirable state.
Let’s use a highway as an example…
A few general observations, you’ve likely noticed:
- The highway typically slows significantly below 100% utilization (the maximum number of cars that can fit on the road).
- The rate at which the highway slows is non-linear. Traffic will start to slow significantly faster between 80% and 90% utilization that it would between 40% and 50% utilization.
- When accidents occur or a lane shuts down, the effect is much more significant at times of high utilization (rush hour) than at times of low utilization (late night).
- The likelihood of reaching gridlock in a fully utilized highway becomes almost certain if it fills to capacity.
These observations can all be attributed to variation in the process.
In many ways, creating software is very similar to driving along a highway. There are a variety of unknowns, a variety of random events that might occur, continuous minor corrections, and a variety of options in getting to our end goal.
Luckily, software development isn’t the only place that we can look for to get some guidance on how to effectively determine our utilization.The nature of stochastic systems has been studied in many fields (telecommunications, fast-food restaurants, transportation systems, manufacturing, processors, and many others). In many of these fields, the general approach at modeling the variation in the system can be found in the study of queueing theory.
If we look to queueing theory, we can find some level of guidance with the following general capacity model.
This model shows us that utilization in a stochastic system follows a power-law distribution.
As we reach the upper levels of utilization, the nature of the work we are performing will likely lead to intense thrashing and will likely have impact across all areas of the work that we are performing. If anything doesn’t go exactly to plan or any other non-planned items come into play, we are likely to jeopardize our commitment.
Given the rate that these items tend to occur, I recommend not aiming for 100% utilization.
Bottom Line: When I see a team of individuals that have (1) calculated a realistic capacity, (2) applied honest estimates to their tasks, and (3) targeted a maximum capacity of 70% – then I feel good that they have done their best to provide a realistic commitment to achieving the completion of their stories for the iteration.
[ Check out more posts from Ben Carey by visitng his blog - The Sherpa Project ]
DeMarco’s book Slack has some of the best material on topics such as utilization and capacity for taking handling unexpected problems or opportunities.
Also, traditional recommended estimation practice has targeted ~75% as expected top capacity (i.e., 6 hours/day). So that fits closely with your recommendation.
And one thing that wasn’t explicit in your post, but I’m reasonably sure you meant it, was that, no matter how many tasks make up the 70%, there is a limit to how many can be expected to be done in parallel due to task switching loss. So that’s another factor for capacity considerations. (And another of the things DeMarco discusses.)
Hi,
Good post. We use a capacity of 6 hours per day although the working day is set to 7.5. That is, a capacity of 80%. We found that it works better than when we set the capacity to the “legal” 7.5 hours. It is nice to read some mathematical evidence of why this works. It is now easier to justify what we do.
Thanks
Marta
Thanks for the comments.
In looking back over the post, I never made it explicit – but the “Working Hours Available per Day” in the capacity calculation might be 8 or it might be 6. I let the team members decide this themselves.
I tend to apply the 70% rule-of-thumb after the capacity is calculated.
In other words…
6 hrs. per day * 8 working days = 48 hours of capacity.
After that capacity calculation, I will be most comfortable with 48 * 70% as a commitment. In this case 33 or 34 hours.
And I do recommend that each team member only works on one task at a time and works each item through to completion (rather than parallel work on multiple tasks).
[...] Stop Over-Committing! – Rethinking Utilization Limits Agile Blog: Scaling Software Agility (tags: agile) [...]
Ben,
Thanks for the post. I remember a colleague in a previous company who was a real extreme programming pro. With regard to this capacity wrangle with new teams, he would not allow anyone on the team to commit over 4 hours per working day until they proved to him that they actually had more ideal hours per day than 4. They would then move to 5 hours per working day. Again, team members were not allowed to commit beyond 5 hours a day until they proved that they could meet commitments and actually be under-utilized.
While I have never been able to convince a new team to do this, I really like the idea of starting with a very low number in order to drive the point home. My usual tactic is to try to hold the team’s estimate of capacity to around 80% and then work backward from there based on whether they didn’t meet their commitment. Or, they met the commitment but with something of a death march regimen in order to do so.
Has anyone else been able to convince a team (and management) to work your way up from a very low number for capacity?
Jean
I have been using this model for a while now when trying to estimate and budget a project as well as plan the first few sprints.
As Jean’s colleague does, I set our capacity at 50% for all new teams and new team members. In every case to date, teams have had this percentage or slightly greater and increasing thereafter by upwards of 10% each iteration thereafter and stabilizing around 70%. That left over 30% is usually absorbed with meetings and understanding requirements of the project work.
New teams have usually hit their maximum potential for Sprint 4 for us when done in 2-week iteration periods and we usually switch to velocity based planning for Sprint 4.
I have actually created a spreadsheet which calculates all this for me and all new projects formed having new teams are estimated using this method.
It is rare that we get the same team on different projects, so capacity has been a real winner for us.