Agile Development Blog

This post has been a long time coming. I’ve started and restarted it at least a half-dozen times.

Quality – there may be no more multi-faceted and powerful attribute in successful software development. Quality is central to everything we do and seek.

• Higher quality leads to greater productivity, throughput and velocity
• Higher quality leads to increased responsiveness, reduced cycle-times, shorter lead-times
• Higher quality leads to improved customer satisfaction, employee satisfaction
• Higher quality leads to better predictability, reduced risk, improved decision making

Or at least that’s my hypothesis…

And that hypothesis is widely shared amongst the Agile and product development communities.  We’ve developed numerous principles, practices and techniques intended to improve quality:  Test Driven Development; Continuous Integration; Automated Build and Deploy; Pair Programming; Customer Demos; Behavior Driven Development; Acceptance Test Driven Development; and Set-based Design techniques are all at least partially focused on yielding quality improvements.

But quality can’t simply be viewed as a set of tools and techniques – independent variables/levers which we hypothesize will lead to improved business outcomes. Quality is also a business outcome unto itself.

This series emphasizes the need to focus on business outcomes (success) first – methods and practices second.  So, putting aside the methods and good practice assumptions of Agile, and focusing solely on the business outcome of improved quality:

Quality = Fewer Defects in Production

We apply Agile quality practices and techniques, because we believe that doing so will yield improved business outcomes – quality (fewer defects in production) being one of those outcomes – along with productivity, predictability, responsiveness, customer and employee satisfaction.

Large, manual, end-of-cycle execution of formal testing by an independent QA organization is also a method aimed at improving these business outcomes. I hypothesize that it is less effective than alternative Agile techniques. But I don’t take that on faith, and neither should you. We must test our hypothesis.

How Do We Measure Quality?

There are innumerable quality metrics that have been devised over the years – each with its own strengths and weaknesses.  In my experience, it’s important to keep metrics simple, and to not let great become the enemy of good enough. In other words, if you have a metric that does a good job of providing insight into the quality of your product/solution, and is simple to collect and interpret; that is likely better than chasing after a metric that will do a great job, but would be more complicated.

For my part, I’ve had success over the years using a couple relatively simple metrics:

• DEFECT DENSITY – # Defects / KLOC
• DEFECT ARRIVAL – # Defects Identified / Month

What Do We Consider a Defect?

In both cases, I include only defects in the production system.

Measuring defects found and eliminated during the development cycle may be useful for managing your development and quality processes.  But from a business outcomes perspective, our focus is reducing the number of defects that make it to production – not making assumptions about how or when to achieve that.

Not All Defects Are Created Equal

Any good metric should drive in more questions than answers.  I find it useful to tag defects with information about type and severity, so that we can consider some of those questions more deeply.

• Our defect density is high; but our severity 1 & 2 density is low. What is the impact on other outcomes (productivity, customer satisfaction, etc.) if we were to invest in reducing our low severity defects?
• Our defect arrival is very high immediately following a major release. But the defects are mostly Type = Usability. Why are our customers having such a tough time using our new features; and how can we ease them through the learning curve?

You may have some hypotheses based on these questions. Perhaps those hypotheses involve application or improved use of Agile tools and techniques. What experiments would you run to prove or disprove your hypothesis?  What new questions will those results yield?

This is the third post in our blog series, Measuring the Impact of Your Agile Investments. The series focuses on measuring the impact that Agile practices have on business outcomes.

Isaac Montgomery is the harried father of twin sons, a frustrated hack on the golf course, and an Agile Coach at Rally Software. He blogs at Leading Results, you can follow him on twitter at @iwmontgomery

We have all felt the pull of game play mechanics in software. You might be addicted to Angry Birds, Farmville, Foursquare or Mafia Wars? Or, maybe like me, you felt compelled to ski a couple extra runs this year thanks to the Epic Mix from Vail Associates. In either case, the achievement leveling and badging associated with the “gamification” of this software has most likely had some impact on your behavior.

Nowhere have I seen these techniques applied to software like I have experienced in StackExchange, a network of Q&A sites founded by Jeff Atwood and Joel Spolsky. Most of my experience is with the Project Management StackExchange, but there are 51 public sites and over 50 other domains emerging. Thanks to smart work by the StackExchange team, the leveling and badging mechanics are used to pull you into an ownership position with the community. As you earn reputation points, you are granted more privileges on the site. This progressive enablement of editing, voting, chatting and commenting capabilities seems perfectly matched with my gaining experience of the culture and ethics of the site. The more I use the site, the more I find myself developing a real sense of ownership and responsibility to the community. This is simply beautiful software for building a community of experts.

My positive experience with StackExchange has been echoed by a bunch of others at Rally. In fact, after playing with site back in March, Rally decided to partner with StackExchange to help share the knowledge from our inaugural RallyON Conference. Specifically, we started working with the project management and programers sites, as they have good coverage of agile, lean software, scrum, kanban, test-driven development, and continuous integration topics.

I encourage you follow our lead and try out StackExchange personally and with your agile teams. I think you will find it to be a great community for capturing and sharing knowledge on agile. Don’t miss Jean’s recent post, “Life in the StackExchange Lane,” to hear about her first month with the site.

Click to register for the webinar- Defining Done

For us, StackExchange is quickly becoming an indispensable community building tool – let me tell you the story and why we are going to use it to clear questions for the next event in our agile webinar series! To get started, please see this example question on pm.stackexchange.com – “How do you define “Done” on a project?” To see how the StackExchange community is preparing for this experiment, you can view the question - “Growing the site with a new experiment” in the meta section of pm.stackexchange.com.

To understand the rational for all this work I want to explore three areas: First, recognizing what was not working for us in our community;  Second, appreciating the stack overflow approach behind StackExchange; Third, comparing and contrasting StackExchange with other Q&A sites.

It’s hard to build a general community, but we need to

Since 2004, we have been a provider of agile solutions through the combination of products and services. For our customers, Rally and its partners deliver large and sustainable gains in software development time-to-market, quality and productivity as well as increasing the sense of purpose and joy on teams. To increase the impact of agile for our users who are spread arcross 100 countries, we launched a social community in 2006; Agile Commons.

Agile Commons provided an open platform to encourage dialogue and discussions with our users and others in the community. Of course there are many places on the Internet to have these general discussions. As a result, the parts of Agile Commons that really took off were those more closely associated with Rally specific content. We just did not have enough traffic to clear the questions with well thought out answers that really covered a problem space. As a result, Agile Commons has morphed into an open commons primarily for Rally customers and users. In addition, the general Agile community discussion has continued to splinter across countless blogs (see the top 200 agile blog list – we are #12!), email lists, and twitter. Due to this splintering, it is really hard to quickly find good, well shaped answers to common agile questions.

This problem has been plaguing the agile community for years and finally boiled to the surface at the 10 year agile gathering in Snowbird this year. In that meeting the following four items were cited as critical steps to keep the community growing for the next 10 years:

• Demand Technical Excellence
• Promote Individual Change and Lead Organizational Change
• Organize Knowledge and Improve Education
• Maximize Value Creation Across the Entire Process

You can read more about the 10 years agile gathering in my February post as well as the many sites and attendees that I reference. I think the industry is ready to address this problem. Now what is the solution?

StackOverflow thinking

I have been passionate about building and sharing knowledge since I was first introduced to web technology via Mosiac in 1994; however, I would not call myself a knowledge management expert. I have continued to dip in and out of this space but being introduced to David Snowden’s work at the Lean Conference in 2010 has been a significant catalyst in my thinking and passion on social and knowledge management. His work has stoked my fire around this problem and solution space. David’s talks and the morphing of Agile Commons have driven my pursuit of a great space to manage agile knowledge in an open manner. My research took me through:

• Wikipedia on agile
• Agile Alliance Resources
• Lean Software & Systems Wiki
• Scrum Alliance community
• Linked In Groups on agile
• Google’s Knol on agile
• Quora on Agile
• Stackexchange on project management

It was StackExchange that stood out to me as the clear winner for managing what Snowden calls ordered knowledge. StackExchange’s Q&A format is truly amazing, it is first a community of experts and second a well gardened knowledge management system. See the PM StackExchange ABOUT post to understand how it is a combination of four great technologies.

If you have not tried stackexhange, jump into pm.stackexchange.com and try entering any project management question you can think of, including anything agile. As you type, you should see a list of related questions based on the keywords in your question. If you do not see your question, please enter it using these simple guidelines and make sure to use tags like agile, scrum, kanban, or TDD. The community will help you shape it into something that will get a good spectrum of answers in a matter of a week. Even in beta the PM StackExchange includes the following site stats:

I don’t care what yahoo group or wiki you are on in our community, it’s difficult find that kind of diverse network to help you with your day to day questions. As I noted, the site is still in public beta. My guess is by 2012, this community will have quadrupled.

Problems with other Q&A sites

This post is about Stackexchange, but, as I mentioned above, there are other solutions for managing a body of knowledge like this. I found a number of short-comings in those communities:

• There is not enough people in community to clear the answer broadly and quickly – too small a sample
• There is only a certain clique of people in a community that provides too much of a myopic answer – 1 right way
• There is focus on discussing and debating, not answering the question in a focused way that matches the question depth – a podium
• There is opacity with regard to governance and content ownership – lack of transparency = low trust
• There is a lack of moderation to keep the community – entropy happens

I believe StackExchange addresses all these issues in a remarkable set of people, policies and bots. I encourage you to help our community move forward by finding ways to organize and share knowledge on Agile in StackExchange. Please share your ideas and other agile resources in the comments.

Ryan Martens is CTO/Founder of Rally and on the way to be the Entrepreneur-in-Residence at the Unreasonable Institute this summer in Boulder –  See the Institute’s 2011 Fellows – Watch the intro video to the Institute and follow my escapades in the Unreasonable Mansion with twitter @RallyOn

Here’s something obvious I’ve learned the hard way: delivering “potentially shippable product increments” each and every Sprint isn’t easy.  Essentially you’re trying to take all the disparate activities that occur throughout the waterfall process, focus them on just the little product increment’s functionality and then jam them into a teensy little Sprint.  Hard to do and definitely pretty unlikely to get accomplished right out of the chute. The authors of “Practices for Scaling Lean & Agile Development”, Craig Larman and Bas Vodde, forgive you in advance for not getting this done straight away. In fact they suggest that this is more a goal the team will approach over time than a rule they put in place on day one.

The authors propose something very simple but very insightful:

• Sketch out all the things you need to do to get your product out the door.
• Define “done” as that subset of those the team is currently capable of performing every Sprint.
• Use your retrospectives to challenge the team to bring more and more of the “undone” work into each Sprint.

This has already changed the way I think about retrospectives. For other nuggets from Larman and Vodde’s book, read on.

Overall a Must-Read for Agile Development Leaders

I was blown away by “Scaling Lean & Agile Development”, as you can see from my bullish review on O’Reilly.  Some time has passed since then but I still feel that it’s one of the most important development books I’ve read.  That book alluded to the companion volume, “Practices for Scaling Lean & Agile Development”, and as you can imagine I awaited its publication eagerly.  It came out in February – I’ve worked my way through it now.   It’s most definitely a worthy successor.

The first book presents theoretical and philosophical underpinnings for agile and lean development. The second book presents a survey of practices relevant to all aspects of the process of developing software at scale, presented by two guys who bring a wealth of knowledge and experience to the table.

Continual Investment in Requirements

Larman and Vodde present practices relevant to a continual investment in requirements throughout the product development process. This is done both up-front, in seeding the product backlog, and iteratively, in refining requirements to support upcoming Sprints.

I love the emphasis they place on whole-team involvement in the initial Product Backlog development effort – even for projects with large teams.  Too often I’ve seen this be the provenance of Product Owners, working in near isolation, which does little to get the project off on a good footing.  The first the team sees of the requirements is the Product Backlog itself, having been involved in none of the  discussions and thought that went into it.

The notion of requirements areas, which was introduced in the first book, is leveraged here to help parallelize the initial requirements development work.  Concurrent sizing and value estimation workshops keep the requirements work rooted in reality.  They spend some time on the problem of bootstrapping a consistent sizing process in a large scale team. The use of cross-team estimation sessions result in the development of a canonical set of sized stories used as a basis for subsequent sizing at the team level.

In Favor of Forward-Looking Requirements Refinement

The discussion of forward-looking requirements refinement really resonated with me. I’ve found in my own practice that peeking ahead at the Product Backlog items upcoming in the next Sprint and then spending time together working through them to understand what they really mean – before we get into the Sprint planning session – goes a long way towards supporting a predictable iterative process.

It also puts the requirements elaboration just barely outside the Sprint that the work is planned for.  This separates coming to an understanding about what we want from the stress of figuring out how to fit it into a Sprint, which makes for more open and enjoyable requirements development.  The authors suggest the use of examples and Acceptance Test Driven Development (ATTD) to more precisely capture the intended behaviors of the stories in a way that realizes both conversation and confirmation of the “three Cs” . Smart.

The authors show how traditional approaches to project scoping and commitment (where separate product management and development organizations act essentially as competitors) are very much analogous to the contract negotiation that the Agile Manifesto cautions us against.  I’m surprised to admit this never occurred to me – I always read that part of the manifesto pretty literally. But it’s inarguably true that the wrangling between these two groups over project scope and timelines that happens in many organizations is a form of contract negotiation, and the waste it drives can be startling.

The arm wrestling of the product management “market ask” captured in a Market Requirements Document followed by the team’s “development response” carefully defined in a Product Requirements Document is a case in point.  Weeks and months can go by during this ritual.  What are the development teams doing during this time?  Not at lot, in my experience.  Does development really “win” if they manage to push out the end date or reduce scope?  Does product management really “win” if they manage to squeeze in extra features or pull in the release date?  I’ve seen how the empowerment of the Product Owner and the establishment of the Product Backlog as the single high-level requirements and release scoping artifact helped to prevent some of these painful dysfunctions but I’ve never thought of them in terms of an imbalance of contract negotiation over customer collaboration.

Creating “Desire Lines” in Design

Larman and Vodde strongly advocate wikis as the preferred basis for the technical documentation the team develops.  They suggest a page for the overall product and pages for each Sprint. That worked for my teams as well, although there was always a certain (healthy) tension regarding what was appropriate for Sprint documentation and what should be living documentation at the product level.

They present the compelling idea of “desire lines”, which, in landscaping, refer to paths that develop naturally, as people use a given space.  Rather than guessing how people would use the area, they are observed using it and then the landscaping is designed around their actual usage.  Similarly in design, rather than trying to guess what the needs are, let them emerge and then refactor to support the emerging design tensions.  A lovely analogy, I thought, and one that will stay with me.

They suggest both a priori design in collaborative design workshops and design after the fact in System Architecture Documentation (SAD) workshops.  I like the acceptance that both approaches are going to be useful. The former stresses the need for team contribution to the design as the Product Backlog items are being developed, while the latter recognizes that team needs for technical documentation will emerge over time and so setting time aside to fine tune design documentation is both warranted and healthy.

The authors also address the question of whether and when to rewrite code – like Joel Spolsky, I personally favor refactoring to improve legacy code rather then re-engineering.  The authors present a compelling and complementary argument for refactoring and incremental improvement based on the value of instilling a notion of continual (rather than punctuated) improvement in the development teams.  They stress that the real problem isn’t the legacy code you’ve got but rather the legacy code you continue to write.  Encouraging the team to have a mindset of each check-in leaving the code base better than it was before – fixing the broken windows and taking out the trash – is a better solution to the problem of poor code quality than is carving off large sections of time for exclusively improvement-oriented work.

Acceptance Test Driven Development

The book’s worth it just for this material alone.  Larman and Vodde present a wonderful analysis of ATDD and characterize its place in the context of Scrum, including tying it into iterative requirements refinement, the Definition of Done and the Sprint Review.  I’d seen teams go in this direction driven largely by the desire to better engage testers in their teams throughout the Sprint and avoid “scrummerfall.” The authors take this further to show how ATDD, in which acceptance tests are defined and automated in advance of the development of the code that will pass those tests, does for teams in an iteration what Test Driven Development (TDD) does for individual developers in ten minutes.

They stress the need for the test automation to be a distributed responsibility shared by the whole team rather than one assigned to a specialist team.  I couldn’t agree more.  I’ve seen many attempts to establish test automation through the creation of a group apart from development and I can’t say that I’d call any of them particularly successful – at least not in the broad and encompassing sense that Larman and Vodde are envisioning in this book.

Continuous Integration at Scale

I was particularly glad to see treatment of Continuous Integration (CI) at scale. I’ve seen groups start with vanilla CI as it is described in the Extreme Programming literature and do well with it initially but then fail as their groups grew larger.

Essentially, Larman and Vodde propose a nested set of continuous integration builds, each larger one subsuming sets of smaller ones within it and each build defined for a particular concern.  Examples of these concerns include the verification of specific components and subsystems but also particular kinds of testing – including things like performance and stability testing.  One key aspect of this approach is that, at each level, you’re trading off the immediacy of feedback to the developer against the likelihood of the developer’s submission affecting quality at that level and the expense of the tests.

The Elusive Definition of Done

As noted earlier, Larman and Vodde,  suggest defining “done” as that subset of the work needed to release the product the team is currently capable of performing each and every Sprint – then use your retrospectives to challenge the team to bring more and more of the “undone” work into each Sprint.  The authors point out that there are really only two ways to extend the Definition of Done:  increase the cross-functionality of the team or increase the degree of automation the team uses.

In the meantime, however, while the Definition of Done leaves some work undone, Larman and Vodde suggest meeting that problem honestly, by doing things like defining an undone work team and pushing undone work onto the Product Backlog.  By being explicit about where the team currently stands against the goal of delivering potentially shippable product increments, this undone work can be better managed and the team’s insights can be directed to the problem of expanding that definition of done to get them closer to the goal.

For the Bookshelf of any Agile Team Member

The book isn’t without its faults.  It’s certainly long enough and some sections can be tedious (there’s a fifteen or so page section where different scenarios for story splitting are laboriously addressed – I get that this common complaint about stories needs to be put to rest but this feels like killing me with kindness :)).  The book’s organization lends itself more to use as a reference than as something you’d read cover to cover.   There are many repetitive sections that allow each chapter to stand on its own but are a bit hard to get through when you read it straight through.  But these are really just quibbles.  In all, “Practices for Scaling Lean  & Agile Development” is, just as its companion volume before it was, a tremendous book that belongs on the bookshelf of any agile coach, manager or team member.

I’d like to thank Anne Greenhaw and Selaine Henriksen for their help in developing this post.  Thanks also to Ryan Martens for inviting me to post here.

About the Author: Ed Willis has been a ScrumMaster, Product Owner, Scrum coach and trainer.  He is currently a developer in the telecommunications industry.

Question: How did you get started from waterfall?

Answer 1 -  Started with whiteboards and sticky notes.

Answer 2 – Started with in-house tool, then open source and finally Rally.

Answer 3 – Started with Rally training for the first year with a Rally pilot in parallel to show management and staff the whole solution.

Answer 4 – Put good consultants alongside the team from day 1 to apprenticeship with the best – we went Rally training and consulting.

Question: We have lots of concurrent features in parallel. Does anyone do concurrent feature branches?

Answer 1 – Peggy – At Avaya we do this, but first go back to the product owner to prioritize better.  We use private views in ClearCase to manage features branches with continuous integration in the views.

Answer 2 – Israel – Never allow the backlog to be larger than twice the capacity of a release.

Question: Please talk about the adoption challenges of adopting with your external customers.

Answer 1 – Israel – It took us 18 months to change the perception of my group with sales.  It was critical to get field engineers to come to release planning and every iteration demo. They get exposed to the software engineering approach and real status and built the best evangelists and advocates in the process.

Answer 2 – Richard – Rally has developed a social networking site with what most people see as shocking openness.  As a result, we have had tremendous success in engaging customers and focusing priorities.

Question: What is the cheapest and best way to get started as an individual?

Answer 1 – Take trips to visit other local companies – everyone says yes.

Answer 2 – “XP in Trenches” – was recent great book find- read a ton of books but most only focused on single team.

Answer 3 – Richard – Come be a fly on the wall at a Rally release planning.

Question: Big Six Sigma background – What tools do you use QFD to find what customer really wants?

Answer 1 – Failed with all upfront tools, only thing that works is communication in every iteration.  Too many edge cases in our industrial software.  Given up trying to spec this in advance.  Invest more in bringing the customer in.

Answer 2 – Israel – Try to reach the point where the developer can code faster than marketers can release.  As soon as that happens, you can decouple and get more feedback during the cycle.  You are also able to bring more of the business into learning after you decouple.

Question: How do you solve the coordination of many teams in a large adoption? Tell me more about Meta Scrum.

Answer 1 – We do a meta scrum every week with the owners of seven product lines.  In parallel the product owners and development leads do the same.

Answer 2 – Israel – Scrum of Scrums everyday with all the ScrumMasters following a 10 to 15 minute approach.