Agile Development Blog

This post has been a long time coming. I’ve started and restarted it at least a half-dozen times.

Quality – there may be no more multi-faceted and powerful attribute in successful software development. Quality is central to everything we do and seek.

• Higher quality leads to greater productivity, throughput and velocity
• Higher quality leads to increased responsiveness, reduced cycle-times, shorter lead-times
• Higher quality leads to improved customer satisfaction, employee satisfaction
• Higher quality leads to better predictability, reduced risk, improved decision making

Or at least that’s my hypothesis…

And that hypothesis is widely shared amongst the Agile and product development communities.  We’ve developed numerous principles, practices and techniques intended to improve quality:  Test Driven Development; Continuous Integration; Automated Build and Deploy; Pair Programming; Customer Demos; Behavior Driven Development; Acceptance Test Driven Development; and Set-based Design techniques are all at least partially focused on yielding quality improvements.

But quality can’t simply be viewed as a set of tools and techniques – independent variables/levers which we hypothesize will lead to improved business outcomes. Quality is also a business outcome unto itself.

This series emphasizes the need to focus on business outcomes (success) first – methods and practices second.  So, putting aside the methods and good practice assumptions of Agile, and focusing solely on the business outcome of improved quality:

Quality = Fewer Defects in Production

We apply Agile quality practices and techniques, because we believe that doing so will yield improved business outcomes – quality (fewer defects in production) being one of those outcomes – along with productivity, predictability, responsiveness, customer and employee satisfaction.

Large, manual, end-of-cycle execution of formal testing by an independent QA organization is also a method aimed at improving these business outcomes. I hypothesize that it is less effective than alternative Agile techniques. But I don’t take that on faith, and neither should you. We must test our hypothesis.

How Do We Measure Quality?

There are innumerable quality metrics that have been devised over the years – each with its own strengths and weaknesses.  In my experience, it’s important to keep metrics simple, and to not let great become the enemy of good enough. In other words, if you have a metric that does a good job of providing insight into the quality of your product/solution, and is simple to collect and interpret; that is likely better than chasing after a metric that will do a great job, but would be more complicated.

For my part, I’ve had success over the years using a couple relatively simple metrics:

• DEFECT DENSITY – # Defects / KLOC
• DEFECT ARRIVAL – # Defects Identified / Month

What Do We Consider a Defect?

In both cases, I include only defects in the production system.

Measuring defects found and eliminated during the development cycle may be useful for managing your development and quality processes.  But from a business outcomes perspective, our focus is reducing the number of defects that make it to production – not making assumptions about how or when to achieve that.

Not All Defects Are Created Equal

Any good metric should drive in more questions than answers.  I find it useful to tag defects with information about type and severity, so that we can consider some of those questions more deeply.

• Our defect density is high; but our severity 1 & 2 density is low. What is the impact on other outcomes (productivity, customer satisfaction, etc.) if we were to invest in reducing our low severity defects?
• Our defect arrival is very high immediately following a major release. But the defects are mostly Type = Usability. Why are our customers having such a tough time using our new features; and how can we ease them through the learning curve?

You may have some hypotheses based on these questions. Perhaps those hypotheses involve application or improved use of Agile tools and techniques. What experiments would you run to prove or disprove your hypothesis?  What new questions will those results yield?

This is the third post in our blog series, Measuring the Impact of Your Agile Investments. The series focuses on measuring the impact that Agile practices have on business outcomes.

Isaac Montgomery is the harried father of twin sons, a frustrated hack on the golf course, and an Agile Coach at Rally Software. He blogs at Leading Results, you can follow him on twitter at @iwmontgomery

We have all felt the pull of game play mechanics in software. You might be addicted to Angry Birds, Farmville, Foursquare or Mafia Wars? Or, maybe like me, you felt compelled to ski a couple extra runs this year thanks to the Epic Mix from Vail Associates. In either case, the achievement leveling and badging associated with the “gamification” of this software has most likely had some impact on your behavior.

Nowhere have I seen these techniques applied to software like I have experienced in StackExchange, a network of Q&A sites founded by Jeff Atwood and Joel Spolsky. Most of my experience is with the Project Management StackExchange, but there are 51 public sites and over 50 other domains emerging. Thanks to smart work by the StackExchange team, the leveling and badging mechanics are used to pull you into an ownership position with the community. As you earn reputation points, you are granted more privileges on the site. This progressive enablement of editing, voting, chatting and commenting capabilities seems perfectly matched with my gaining experience of the culture and ethics of the site. The more I use the site, the more I find myself developing a real sense of ownership and responsibility to the community. This is simply beautiful software for building a community of experts.

My positive experience with StackExchange has been echoed by a bunch of others at Rally. In fact, after playing with site back in March, Rally decided to partner with StackExchange to help share the knowledge from our inaugural RallyON Conference. Specifically, we started working with the project management and programers sites, as they have good coverage of agile, lean software, scrum, kanban, test-driven development, and continuous integration topics.

I encourage you follow our lead and try out StackExchange personally and with your agile teams. I think you will find it to be a great community for capturing and sharing knowledge on agile. Don’t miss Jean’s recent post, “Life in the StackExchange Lane,” to hear about her first month with the site.

Click to register for the webinar- Defining Done

For us, StackExchange is quickly becoming an indispensable community building tool – let me tell you the story and why we are going to use it to clear questions for the next event in our agile webinar series! To get started, please see this example question on pm.stackexchange.com – “How do you define “Done” on a project?” To see how the StackExchange community is preparing for this experiment, you can view the question - “Growing the site with a new experiment” in the meta section of pm.stackexchange.com.

To understand the rational for all this work I want to explore three areas: First, recognizing what was not working for us in our community;  Second, appreciating the stack overflow approach behind StackExchange; Third, comparing and contrasting StackExchange with other Q&A sites.

It’s hard to build a general community, but we need to

Since 2004, we have been a provider of agile solutions through the combination of products and services. For our customers, Rally and its partners deliver large and sustainable gains in software development time-to-market, quality and productivity as well as increasing the sense of purpose and joy on teams. To increase the impact of agile for our users who are spread arcross 100 countries, we launched a social community in 2006; Agile Commons.

Agile Commons provided an open platform to encourage dialogue and discussions with our users and others in the community. Of course there are many places on the Internet to have these general discussions. As a result, the parts of Agile Commons that really took off were those more closely associated with Rally specific content. We just did not have enough traffic to clear the questions with well thought out answers that really covered a problem space. As a result, Agile Commons has morphed into an open commons primarily for Rally customers and users. In addition, the general Agile community discussion has continued to splinter across countless blogs (see the top 200 agile blog list – we are #12!), email lists, and twitter. Due to this splintering, it is really hard to quickly find good, well shaped answers to common agile questions.

This problem has been plaguing the agile community for years and finally boiled to the surface at the 10 year agile gathering in Snowbird this year. In that meeting the following four items were cited as critical steps to keep the community growing for the next 10 years:

• Demand Technical Excellence
• Promote Individual Change and Lead Organizational Change
• Organize Knowledge and Improve Education
• Maximize Value Creation Across the Entire Process

You can read more about the 10 years agile gathering in my February post as well as the many sites and attendees that I reference. I think the industry is ready to address this problem. Now what is the solution?

StackOverflow thinking

I have been passionate about building and sharing knowledge since I was first introduced to web technology via Mosiac in 1994; however, I would not call myself a knowledge management expert. I have continued to dip in and out of this space but being introduced to David Snowden’s work at the Lean Conference in 2010 has been a significant catalyst in my thinking and passion on social and knowledge management. His work has stoked my fire around this problem and solution space. David’s talks and the morphing of Agile Commons have driven my pursuit of a great space to manage agile knowledge in an open manner. My research took me through:

• Wikipedia on agile
• Agile Alliance Resources
• Lean Software & Systems Wiki
• Scrum Alliance community
• Linked In Groups on agile
• Google’s Knol on agile
• Quora on Agile
• Stackexchange on project management

It was StackExchange that stood out to me as the clear winner for managing what Snowden calls ordered knowledge. StackExchange’s Q&A format is truly amazing, it is first a community of experts and second a well gardened knowledge management system. See the PM StackExchange ABOUT post to understand how it is a combination of four great technologies.

If you have not tried stackexhange, jump into pm.stackexchange.com and try entering any project management question you can think of, including anything agile. As you type, you should see a list of related questions based on the keywords in your question. If you do not see your question, please enter it using these simple guidelines and make sure to use tags like agile, scrum, kanban, or TDD. The community will help you shape it into something that will get a good spectrum of answers in a matter of a week. Even in beta the PM StackExchange includes the following site stats:

I don’t care what yahoo group or wiki you are on in our community, it’s difficult find that kind of diverse network to help you with your day to day questions. As I noted, the site is still in public beta. My guess is by 2012, this community will have quadrupled.

Problems with other Q&A sites

This post is about Stackexchange, but, as I mentioned above, there are other solutions for managing a body of knowledge like this. I found a number of short-comings in those communities:

• There is not enough people in community to clear the answer broadly and quickly – too small a sample
• There is only a certain clique of people in a community that provides too much of a myopic answer – 1 right way
• There is focus on discussing and debating, not answering the question in a focused way that matches the question depth – a podium
• There is opacity with regard to governance and content ownership – lack of transparency = low trust
• There is a lack of moderation to keep the community – entropy happens

I believe StackExchange addresses all these issues in a remarkable set of people, policies and bots. I encourage you to help our community move forward by finding ways to organize and share knowledge on Agile in StackExchange. Please share your ideas and other agile resources in the comments.

Ryan Martens is CTO/Founder of Rally and on the way to be the Entrepreneur-in-Residence at the Unreasonable Institute this summer in Boulder –  See the Institute’s 2011 Fellows – Watch the intro video to the Institute and follow my escapades in the Unreasonable Mansion with twitter @RallyOn

The LA Agile Success Tour concluded with 5 open space sessions. For my other live posts from the LA event, see:

• 5 Stories of Agile Success
  
• What Would You Ask Your CEO to Bring in Agile?

Did you find these posts valuable?  Any advice on how we can improve this for next time?

Agile 101, led by Christophe Louvion

What is your pain? Command and control, changing priorities during the sprint, hybrid process, making decisions outside the team that affect the team, improper tooling, moving resources around.

Root causes: Lack of self organization caused by lack of education, lack of an Agile driver, internal or external customer pressure, lack of executive vision, fear of change, skill set mismatches, ask teams to do things that don’t work well, lack of action items with a clear owner and clear deadlines.

Christophe had people in his breakout commit to action items. Christophe is going to call each of these people and verify if they accomplished their action items, like Lisa who committed to requesting a coach to come into their company and help them with their hybrid Agile process.

Get something small, but get it done!

Socializing Agile, led by Israel Gat

Pains: The sausage factory syndrome, the business doesn’t want to know how the software is made, but give me what I ask for when I need it. Market dynamics like drops in ad revenue and other drops in the market.

Solution: Innovation is now affordable because we can all experiment, try something in your sprint and if it doesn’t work out you only have 2 weeks invested. Read the book Innovators Dilemma by Clayton Christenson.

Risk management is important to executives, and Agile reduces it significantly. Money quantification, understand the value of what you’re delivering. Look at Capers Jones and his 2008 book Applied Software Measurement and his 2007 book Estimating Software Costs.

Agile Governance and Compliance, led by Laureen Knudsen

Pains: People live in rigid environments and don’t want to change. People don’t want to show their work because it’s too raw and there is fear around failing fast by validating your work with actual customers. There were questions about what is the earliest point to show things to customers.

Agile Collaboration, led by Jean Tabaka

Organizationally what is happening to enable collaboration? Organizations need to communicate down about vision, and the teams need to communicate up areas of process improvement. There was talk about servant leaders, and how command and control managers actually make people dumber. Most Agile teams start with iteration planning, standups and the end of the iteration demo.

Jean has a different view, called the 5 Levels of Planning.

Start with a Vision, this is the first level of planning that the leadership team owns.

Then move to a Roadmap where you talk about themes of releases. It’s important to talk in themes and the Product Council is the feedback mechanism for that process.

Then Release planning is next. It is important for everyone to be involved in the Release planning.

Then think about taking the Release theme and how the Iterations build toward that goal.

Then Daily examine what you did toward that goal yesterday, what you are going to do today, and what is getting in your way.

Each level then provides a feedback loop to the higher level of planning. At Rally everyone is involved in these 5 levels of planning within every department in the company.

Quality and Testing, led by Zach Nies (me)

Pains: Integrating testing teams in with development teams, making testing teams more effective and how to introduce Test Driven Development into the development team. An important way to integrate testing and development is to integrate and involve both roles in planning sessions. Testing should be part of the definition of done and involve testers in creating a story’s acceptance criteria.

The group talked about how to break down the walls between testers and developers, by focusing on the reality that everyone owns the quality of the software being developed. When testing is integrated into the process it makes them more effective. Finally, a few teams are trying to use Test Driven Development. Test Driven Development can’t be imposed onto the team, the key developers on the team need to lead by example by embracing TDD practices.