I would like to welcome Ed Willis to our blog, as a guest blogger. I spent a couple of days with Ed and his team of internal coaches this Winter and I am glad to have him share some of their lessons from the field of large distributed Agile – Ryan
Question: What are we, as Scrum team members, committing to in a sprint?
The higher-level Sprint Goal or the more detailed set of selected Product Backlog? If you are a customer of the team, what is the team telling you it will do?
I recall being pretty confused about what the team was committing to in Sprint planning after reading Agile Software Development with Scrum, by Ken Schwaber and Mike Beedle, for the first time. There seemed to be some ambiguity around whether the Scrum Team commits to the set of selected Product Backlog items or to the Sprint Goal, which is a higher level and less precise concept. A couple of quotes from that book will help illuminate the situation:
“A team commits to achieving a Sprint Goal.”
“The Scrum Team commits to turn a selected set of Product Backlog into a working product.”
And this from Schwaber’s The Enterprise and Scrum: “… the team selects as much Product Backlog as it believes that it can transform into a completed increment of potentially shippable product functionality by the end of the Sprint. The team commits to the Product Owner to do its best to complete that amount of functionality.”
From the first book again, “The Sprint Goal is an objective that will be met through the implementation of the Product Backlog … The reason for having a Sprint Goal is to give the team some wiggle room regarding the functionality … If the work turns out to be harder than the team had expected, then the team might only partially implement the functionality. “
Between the current Scrum Guide and 2001′s “Agile Software Development with Scrum”, Schwaber’s Agile Project Management with Scrum presents an overview of Scrum in its first chapter which does not discuss the Sprint Goal at all – similarly, “The Enterprise and Scrum” provides a summary of Scrum in an appendix that does not talk about the Sprint Goal.
So what are we, as Scrum team members, committing to: the higher-level Sprint Goal or to the more specific set of selected Product Backlog? And why am I so focused on Sprint-level commitments? I focus on commitments because my stakeholders and customers value them. Their plans are in part based on my team’s delivery and so being able to make and meet short-term commitments makes us a better partner to them – “don’t trouble your customer”.
Can You Commit to an Incomplete Sprint?
“Agile Software Development with Scrum” states, “Sometimes only a partial Sprint Backlog can be created … define the initial investigation, design, and architecture work in as much detail as possible, and leave reminders for work that will probably have to be done once the investigation or design has been completed.”
Both that quote and the earlier one on the purpose of the Sprint Goal suggest that the Scrum canon allows for both incomplete Sprint planning and a mechanism to de-scope the Sprint if the team falls behind. If you find yourself in a situation similar to mine (and I suspect many, many of us find ourselves in this situation) where your customers and stakeholders really do value the ability to make and meet high-level commitments in terms of working software that will be delivered at Sprint’s end, then I think that finding a way to do this more regularly is a path you can and should pursue. And it’s one my colleagues and I have embarked on wholeheartedly. All you really need to do to become a better partner in this regard is advance your Sprint planning practices and adapt the strategies you use when you discover you’ve fallen behind.
Reliably Committing to the Selected Product Backlog – One Group’s Perspective
Being able to reliably deliver against Product Backlog commitments isn’t hard conceptually – it starts with doing a better job in Sprint planning. There are probably many ways to do this but I thought I’d share what we’ve done to accomplish this.
Our Sprint planning starts with a detailed view into capacity. Is anyone on the team going to be away on training or vacation during the Sprint? Will the remainder of their time be focused on the Sprint? If not, how much do time they think they’ll have to spend in the Sprint? Knowing the total capacity of the team in the coming Sprint is the foundation of a solid Sprint plan.
Ideal Time Estimation
To estimate our Sprint Backlog tasks, we use “ideal time” as it is described in Planning Extreme Programming by Beck and Fowler: “Ideal time is time without interruption where you can concentrate on your work and feel fully productive.” Ideal time on any given task is in essence the answer to the question “how long would this task take you in a perfect world?” Ideal time is then converted into real time in a very similar manner to how story points are converted into expected duration for stories – you measure the velocity of the team. For example, if ideal time velocity of a team is measured at 0.66, then two ideal days of work has been seen in the past to be about three calendar days.
That use of past experience is critical because it rolls up estimation error, overhead, interruptions et al into a single factor used to improve the accuracy of the task estimates while only requiring of the people doing the estimates that they do so consistently. We’ve found it useful. Conceptually I also like it because, along with the use of story points and their corresponding velocity, it increases Scrum’s fractal, self-similar nature.
Typically we assign all the tasks in the Sprint Backlog during the planning meeting. There’s probably enough divergence of thought on this point to warrant its own post, but to stick to the matter at hand, we do this for reasons relevant to the goal of producing more reliable Sprint plans. One major motivation is the fact that estimates provided by the person who will actually do the work are demonstrably more accurate. The second major motivation is our observation that, when trying to achieve greater work sharing and improve the flow of value to the customer by avoiding bottlenecking through specialists, we do a better job of supporting people who are branching out into unfamiliar territory if we know they’re going to do so a priori. Specifically, we can build time into our Sprints for the people who are more knowledgeable in a given area to mentor or review the work of those who are working in areas less familiar to them.
We make frequent use of spikes to allow us to develop a better or more complete understanding of what a given story will entail thus giving us the insight needed to build more complete Sprint plans.
Reviewing the Sprint Plan
We review our Sprint plans to look for mistakes we commonly make. We’ve gone as far as enumerating the mistakes we make most often to help us look specifically for those as we do Sprint planning. Most of these are different flavors of not applying our “definition of done” and thus ending up with incomplete plans. If your goal is to develop challenging plans but plans that you’re likely to deliver on, then trying hard to ensure your plans are complete is important.
Note that none of the above should be intended to suggest that we value immutable Sprint Backlogs or task assignments. We can and do frequently grow and shrink the Sprint Backlog as the team uncovers more information about what’s needed to deliver the committed Product Backlog. Similarly, we shuffle (and re-estimate) tasks regularly. What we don’t do is allow significant parts of the work to go unplanned or unassigned at the point we decide what we’re committing to deliver.
Beyond Yes or No
Beyond improved Sprint planning, the second critical aspect for us in delivering committed Product Backlog in a Sprint is how we deal with falling behind. I frequently say in delivering Scrum training that if someone comes to you asking if a bunch of features can get done given fixed resourcing by a given date, that the best of all possible answers is “yes”. The worst of all possible answers is “no” – but not because you’re delivering bad news but rather because saying “no” reinforces the fallacy that that decision is really binary in nature and ignores the great value to be gained by digging into the details to see what’s actually possible.
Similarly when Scrum teams fall behind, they frequently see their choices as being to either: stick with the plan or remove lower priority Product Backlog in consultation with the Product Owner. Those are reasonable choices but I much prefer a third option: work very closely with the Product Owner to find an easier way to deliver the committed Product Backlog items in the time remaining – essentially, use the combined talents and insights of the Product Owner and the team to find simpler, cheaper ways to get the selected Product Backlog done. Honestly, it can feel like magic when it happens.
We once had a Sprint on a tool development project that had fallen far behind – it appeared that no amount of plan scrubbing or task shuffling would allow us to deliver the committed Product Backlog by Sprint’s end. We met with the Product Owner with the intention of choosing Product Backlog to remove from the Sprint. In the course of conversation though, the Product Owner got a better feel for the approach the team was taking in developing one specific item – the item that was driving most of the over-run. Collectively, they realized that the approach was fancier than was really required and in short order sketched out an alternative. We immediately revised our Sprint Backlog to remove the old approach and built a plan for the new one – in about an hour, we went from hopelessly behind with no chance of delivering the complete set of Product Backlog to being close to on schedule. I think having a strong team desire to deliver the whole of the selected Product Backlog was key to making this happen.
Anecdotally, I can say that my colleagues and I worked hard to improve our Sprint planning and replanning practices and pretty quickly found ourselves doing much better in meeting our Product Backlog commitments. For example, during one ten month stretch, I counted 93% of Product Backlog items delivered on time in the Sprints in which they were planned. This isn’t the only thing that matters but it made a big difference to us and our stakeholders: it gave us confidence in the team.
- Use the Sprint Goal to capture over-arching but incremental project goals if this concept is meaningful to you and if goals like these are more meaningful commitments than would be the selected Product Backlog – but definitely don’t use the Sprint Goal as a crutch to support poor Sprint planning practices.
- Don’t let the Scrum canon hinder your team’s thinking when tuning your process to better satisfy your business needs – the canon wasn’t created to limit your thinking.
- And above all, don’t believe that your team won’t be able to reliably commit to the selected Product Backlog Sprint over Sprint – you almost certainly can and, if your team and your stakeholders value this, then by all means pursue improvements in this area.
I’d like to thank Anne Greenhaw, Selaine Henriksen and Ryan Martens for their help in developing this post. Thanks to Ryan also for inviting me to post here.
About the Author: This is the first guest blog post of Ed Willis. Ed was a soldier, a cab driver, a security guard, a tree planter, an auto glass installer and shop manager, an employee of several bookstores, a house painter, a dish-washer, and a foot messenger, among at least a few other things before buying his first computer. Since then he has worked as a research assistant at Queen’s University, driven a data mining company into the ground and ridden Nortel Networks nearly all the way down. He currently works for company in telecommunications. Born in NY, NY, he now lives in Ottawa, Ontario.