Welcome back to the third post in our seriesAgile Tooling for High Assurance Software. Last week I started by discussing simple examples of how Rally supports Define|Build|Verify in a single Iteration. Let’s dive deeper by reviewing Leffingwell’s December 17, 2010 post, Software Verification and Validation in High Assurance Development: Verfication: SRS and User Stories. In this post he mentions Traceability Paths as a way to Verify a User Story in an Iteration, see below.
“To assure that each new user story works as intended, we’ll have to verify it. To do that, we’ll use three built-in (and semi-automated) traceability paths:
User story to code – path to the SCM record that illustrates when and where the code was changed to achieve the story
User story to code-level unit test (the “white box” test which assures assure the new code works to its internal specifications, also maintained via path to the SCM change set)
User Story to Story acceptance tests (black box testing of the user story to make sure the system functions as intended)”
This post focuses on the last bullet (these bullets aren’t in priority order) since last week’s post already illustrated an example of User Story to Acceptance Test (Verification) traceability in Rally. For many practitioners Rally is not the only tool in your project environment, so let’s move the conversation forward by illustrating how HP Quality Center, for example, can be used to perform Verification Testing on User Stories that are planned, tracked, and managed in Rally. (more…)
Last summer, Rally started a video series called “Chalk Talks“.
I was fortunate enough to have filmed several“Chalk Talk” videos about some of the basics of Agile software development (The Agile Manifesto, Scrum Basics, Iteration Demo and Review Meeting, and other topics).
We also tapped into the wisdom of some of our other Agile Coaches: Julie Chickering, Mark Kilby, and Ken Clyne.
Our Rally Chalk Talks are informal videos, typically 3 – 5 minutes long, intended to provide quick, easy introductions to Agile topics.
Filmed in a short, tutorial format, these videos are great to share with your team as they are getting up to speed on Agile.
To get a feel for our latest work, here’s Rally Agile Coach Ronica Roth in her great Chalk Talk on User Stories. ( during which you can find out why a dog would want his own laptop :D)
Be sure to check out our entire catalog of Chalk Talks and subscribe to our YouTube channel if you’d like to be notified when we publish new videos. We already have two more Chalk Talks queued up: “Kanban and Scrum” and “Agile and Lean”.
As you look through our current catalog of talks, be sure to let us know what other topics you’d like us to cover in future talks.
In this post, I’d like to respond to some questions posed to Johnny and me during the webinar.
JET = Jean Tabaka
What is the measurement of productivity? Is it user stories per lines of code?
JET: We define productivity index with regard to the Michael Mah statistics collected over 7500 products. It is not about lines of code. Rather it is about delivered value. At a very high-level view then, a PI in an Agile team is about the amount of value delivered by a team based on value delivered over the total number of team members.
Michael Mah shows in his studies, which we referenced in our webinar, that Agile teams tend to have a higher ability to delivery value per team member than the norm across all projects he has surveyed. This supports our view that the way to project cut costs is to help team members increase their PI versus cutting team members or off-shoring work.
When you start on Step 1, one team in flow, do you start immediately with 2-week iterations, even if the teams are used to waterfall?
JET: For any team starting its transition to Agile, the important Flow guidance I give is: Choose a timebox, and stick to it. Religiously.
I give guidance that a shorter timebox is preferable to a longer one. This gives the team that moment of pause for inspection and adaption sooner rather than later.
A shorter timebox invites earlier improvements on process. It also affords the Product Owner or customer representative the opportunity to re-prioritize, add, or delete items in their backlog.
Given these parameters, the team commits to what the timebox length will be. So what do I advise? If asked, I tell teams that 2 weeks is a good starting point: shorter time to inspection versus longer wait for improvement opportunities.
How would you apply Agile, and particularly Scrum, to security related software where design and reviews typically take longer than one iteration?
JET: Agile asks us to evaluate how much design and review MUST occur before features can begin to be coded and tested. Your question is all the more important in your context.
Think of it this way: Imagine in a security software environment, you implemented something and then slammed all kinds of vulnerability tests against it to discover its actual vulnerability. Imagine that this could be in addition to, or even as a replacement to, extensive review up front.
I once worked in a security related software environment. We spent months and months reviewing documentation attempting to conduct independent validation and verification about vulnerabilities before any coding began. How I wish we could have just conducted detailed design, code, and test on our riskiest features and applied our hypotheses against these rather than being left to hope our analysis was adequate!
If we want to introduce Agile in a certain project team, what’s the perfect ratio of Agile consultants versus our own personnel?
JET: Perfection is in the eye of the beholder ;-)
I suggest one mentor per team initially. We have also used one mentor for a team of teams, a program. That creates stability and synergy across the teams and encourages all teams to form their own Agile team agreements while also learning from what has worked well with the other teams.
One more thing: Having a mentor at the management/executive-level is also critical. It may be this same person; it may be someone who has more of an executive stature. In the latter case, be sure the two mentors work tightly with one another.
Would you define ‘user stories’? Similar to Use Cases? Difference?
JET: User stories are an expression of a request from the customer or customer representative for a particular function. They are meant to be short, incomplete, and value-based.
They invite conversation and further clarification of completion: “As a < role >, I want to < action > so that I can < benefit >.”
“As a dog, I want access to ‘The Daily Woof” newspaper on my Kindle so that I can flip pages without opposable thumbs.”
This request for action and benefit is elaborated as it rises in rank among all functions requested. We elaborate these in several ways. One way is to start writing acceptance criteria.
Another way might be to elaborate it through a use case that details actor-system interactions, extensions, and so on. A team makes agreements about how function requests grow in clarity; that is, are use cases our best way to declare DONENESS for this request? Or, in our environment, is a simple description and test case sufficient?
How can we get better reporting metrics for roll-up to the executive level? Are there key mash-ups I should/could use?
JET: without digging into specifics with regard to mashups in our Rally tool, I highly recommend looking at a set of metrics we gathered in our whitepaper: “A CIO Playbook for Adopting for Adopting Scrum” co-authored with Ken Schwaber and Dean Leffingwell.
One appendix in the paper guides teams in a quick self-assessment of their Agile uptake. The second offers metrics both teams and executives can use as new measures of success.
Altering how we measure and guide success in Agile has a great impact on the “stickiness” and intended benefits of the Agile rollout. Rally tool mashups simply support real-time access to such measures. This “reporting” style deviates from the traditional “reporting up”. Rather, the metrics we recommend are meant to keep the entire organization informed about what is really happening. This creates organizational ownership of amplified learning and continuous improvement.
Have you integrated user centered design (usability practices) into the Agile development process? If yes, what was your approach? If no, why not?
JET: I believe user-centered design has a place in Agile development. Design emergence doesn’t mean it can only occur in the planning meeting for the next iteration, IMHO.
creates a high-level vision (informed by many sources);
collaborates with the product owners about how to incorporate this across a product roadmap;
provides more detailed guidance about backlog items specific to the user-centered design;
then engages actively within Scrum teams when product backlog items specifically impact user design changes (where she is tester or owner or delivery team depending on the item.)
Do Agile practices only apply to software delivery, or have you ever seen them applied to successful service delivery?
JET: Jeff Sutherland tells a great story about helping his wife apply Scrum in her life as a Unitarian minister. I also have a colleague who uses it in his volunteer organization. And there are people in my Certified ScrumMaster classes from non-software parts of software organizations eager to bring Agile in. Here at Rally, we use Scrum in EVERY department in the company, including at the executive level.
It is already challenging to obtain the time of critical resources (shared among various projects and sustaining). Are every day SCRUM meetings practically possible?
JET: Everyday meetings are faster than having to write them extensive documentation. And, they are less costly than not having them involved at all. Still, teams make agreements about how critical it is for “part-time” experts to be engaged in every meeting. They inspect and adapt.
Is Scrum mostly for R&D projects with more unknowns or is it equally effective on projects with extensive functional and technical knowledge and skills.
In other words, is Waterfall totally obsolete?
JET: Rigid methodologies don’t serve environments where any amount of change can occur. If you have projects that have no change in scope, no mis-interpretations of requirements, no defects in design, no defects found in code test or integration that would require re-work, I see no reason for you to change. I do not know of such projects.
Traditional teams estimate all of their work in one unit type – regular time. These teams then spend a lot of time wondering why their estimates are off and struggling to get more accurate estimates. Many teams are able to bypass this struggle by estimating on two different estimation scales – points and hours.
User Stories in your product backlog should be estimated in terms of their relative size. Imagine writing your product backlog out on index cards, and sorting them in order, least effort to most effort. It should be relatively easy to apply a simple numeric scale to represent relative size. If your first story is at a size of one, and the next one seems like it would take twice as much effort, you’d score that one a 2.
Once you’ve numbered your product backlog like this, you can measure how many of these units you’re getting done each iteration by adding up the numbers for each accepted story. Over time, if you keep your team together, this “velocity” number should stabilize. If your velocity has been about 15 in the past, you can predict that if the team stays the same, you’ll get 15 done each iteration moving forward.
Whether you call these units “days” or “points” or “tomatoes” is irrelevant, as long as you track how many you’re getting done. I like calling them “points” because it’s politically useful to do so. If a 5 person team is getting 10 days worth of work done each week, management might wonder why they’re only 40% efficient. If they get 10 points worth of work done, the efficiency discussion doesn’t come up in this way.
Why use hours for tasks then? Story estimates are basically numbers you pull out of the air, after a couple of minutes of discussion. But when you’re planning tasks at the start of the iteration, you know a lot more – the detailed acceptance criteria, the detailed task list, and maybe even who’s doing the work. Estimating tasks in hours reflects the precision of the estimate (although I try to stick to 1, 2, 4, 8, and 16 hours because you really can’t be more precise).
Consider this pattern when:
Politics lead to management questioning estimates (Points are less questionable)
People are spending a lot of time worrying about estimation accuracy
Be careful with this pattern when:
Teams are unstable and your velocity varies wildly (use T-shirt sizes instead)
There are several problems with traditional requirements formats
They cost a lot of money to produce and yet you can’t sell them to your customers (waste)
They can be monolithic (90 page use cases) and so they’re hard to manage when different pieces have different importance. The desire to make them comprehensive loads extra work onto the development team. (strain)
They take a long time to produce, holding up the development process and delaying the feedback that comes when users get to see working software (wait time)
The more detailed they are, the more the team thinks they’re complete and the less critical thinking gets applied to them, leading to more defects (waste)
User stories are designed to address these issues. A user story represents a small piece of business value that a software team can deliver within an iteration. Here’s one:
As a customer, I want to check my order status online so that I can know when to expect my package.
There’s actually a lot in that little line – we know who the user is, what’s the activity they want to do, and what’s the underlying goal.
Ron Jeffries has said that there are actually three parts of a user story:
Card - the part we wrote in italics above. It’s just a placeholder – a project management mechanism that we’ll use to remind us that at some point we probably should get around to building an order status thingy. This part should be short enough to write on one side of an index card with a sharpie marker. You could even word it “Order Status Checker Thingy” to emphasize that it’s a placeholder, not a requirement.
Conversation - When this card comes up in the stack as the top priority, the team will talk during iteration planning about the acceptance criteria that will need to be met to satisfy the customer or Product Owner that they built the right kind of “Order Status Checker Thingy”. Sometimes the criteria are noted on the back of an index card or in an agile tracking tool.
Confirmation – The detailed acceptance tests that the team writes during the iteration to define exactly how an “Order Status Checker Thingy” should work, in excruciating detail
A user story is unlike regular requirements because the requirements aren’t specified in detail until the tests are written, around the same time as implementation. Benefits of this approach:
No inherent waste. We’re only writing down as much as we need to document our conversations and make sure we’re all on the same page
Encourages discussion. Your developers and testers know a lot about what they’re building. They’re capable of thinking of important details and preventing defects and other bigger mistakes, if the process encourages them
Easy to manage. If you’re used to use cases, break down your scenarios and steps into stories. You can do the important parts early and postpone the less important parts later.
Easy to manage scope creep. If you think of new things to do, create new stories and add them to your list in priority order. Since you’re always doing the most important things first, you’ll end up naturally managing scope in a way you can’t when you’re working off of requirements documents.
(This is a draft – please add comments or edit the pattern itself)
As I watched the Rockies sweep the NLCS I could not help but to think that the Rockies took advantage of Rally’s professional coaching services to help the team achieve success with their agile rollout. Let’s analyze baseball in regards to agile…
During the regular season baseball teams play in 3 or 4 game series, which can be thoughtof as an agile release. Each game in the series can be thought of an individual iteration with a timebox of 27 outs. The user stories don’t really change, a typical baseball iteration has 9 user stories (innings) with 3 story points each. The velocity of the team
really depends on the pitch count. If the pitch count is low we could consider the story to be completed faster than other stories in terms of relative sizing.
Similar to agile, the size of the team is nine players on defensive. These nine players work very closely together to complete each story and focus all of there energy on the current iteration (game). A team member grabs tasks (outs) when they feel they are able to take on more work. Team members regroup in between each inning for a quick stand-up meeting to discuss the next highest priority story (ok this is a long stretch but maybe they have a stand-up).
After the completion of the iteration there is always a retrospective (post game
conference). During the post game press conference (retrospective) the team manager acts as a scrummaster or coach and shields the team from media controversy. The manager is ultimately shielding his team from stakeholders (fans, media, owner) and focusing them on the current iteration (game).
At the end of a game (iteration) the team has completed 9 user stories and the team owner (product owner) either accepts the game or does not accept it (win/lose).
To end this fairly useless analogy we have to consider the post game celebration. Trust me the Rockies had one heck of a release party last night.
RallyDev.com
I give guidance that a shorter timebox is preferable to a longer one. This gives the team that moment of pause for inspection and adaption sooner rather than later.
