Agile Development Blog

Here’s something obvious I’ve learned the hard way: delivering “potentially shippable product increments” each and every Sprint isn’t easy.  Essentially you’re trying to take all the disparate activities that occur throughout the waterfall process, focus them on just the little product increment’s functionality and then jam them into a teensy little Sprint.  Hard to do and definitely pretty unlikely to get accomplished right out of the chute. The authors of “Practices for Scaling Lean & Agile Development”, Craig Larman and Bas Vodde, forgive you in advance for not getting this done straight away. In fact they suggest that this is more a goal the team will approach over time than a rule they put in place on day one.

The authors propose something very simple but very insightful:

• Sketch out all the things you need to do to get your product out the door.
• Define “done” as that subset of those the team is currently capable of performing every Sprint.
• Use your retrospectives to challenge the team to bring more and more of the “undone” work into each Sprint.

This has already changed the way I think about retrospectives. For other nuggets from Larman and Vodde’s book, read on.

Overall a Must-Read for Agile Development Leaders

I was blown away by “Scaling Lean & Agile Development”, as you can see from my bullish review on O’Reilly.  Some time has passed since then but I still feel that it’s one of the most important development books I’ve read.  That book alluded to the companion volume, “Practices for Scaling Lean & Agile Development”, and as you can imagine I awaited its publication eagerly.  It came out in February – I’ve worked my way through it now.   It’s most definitely a worthy successor.

The first book presents theoretical and philosophical underpinnings for agile and lean development. The second book presents a survey of practices relevant to all aspects of the process of developing software at scale, presented by two guys who bring a wealth of knowledge and experience to the table.

Continual Investment in Requirements

Larman and Vodde present practices relevant to a continual investment in requirements throughout the product development process. This is done both up-front, in seeding the product backlog, and iteratively, in refining requirements to support upcoming Sprints.

I love the emphasis they place on whole-team involvement in the initial Product Backlog development effort – even for projects with large teams.  Too often I’ve seen this be the provenance of Product Owners, working in near isolation, which does little to get the project off on a good footing.  The first the team sees of the requirements is the Product Backlog itself, having been involved in none of the  discussions and thought that went into it.

The notion of requirements areas, which was introduced in the first book, is leveraged here to help parallelize the initial requirements development work.  Concurrent sizing and value estimation workshops keep the requirements work rooted in reality.  They spend some time on the problem of bootstrapping a consistent sizing process in a large scale team. The use of cross-team estimation sessions result in the development of a canonical set of sized stories used as a basis for subsequent sizing at the team level.

In Favor of Forward-Looking Requirements Refinement

The discussion of forward-looking requirements refinement really resonated with me. I’ve found in my own practice that peeking ahead at the Product Backlog items upcoming in the next Sprint and then spending time together working through them to understand what they really mean – before we get into the Sprint planning session – goes a long way towards supporting a predictable iterative process.

It also puts the requirements elaboration just barely outside the Sprint that the work is planned for.  This separates coming to an understanding about what we want from the stress of figuring out how to fit it into a Sprint, which makes for more open and enjoyable requirements development.  The authors suggest the use of examples and Acceptance Test Driven Development (ATTD) to more precisely capture the intended behaviors of the stories in a way that realizes both conversation and confirmation of the “three Cs” . Smart.

The authors show how traditional approaches to project scoping and commitment (where separate product management and development organizations act essentially as competitors) are very much analogous to the contract negotiation that the Agile Manifesto cautions us against.  I’m surprised to admit this never occurred to me – I always read that part of the manifesto pretty literally. But it’s inarguably true that the wrangling between these two groups over project scope and timelines that happens in many organizations is a form of contract negotiation, and the waste it drives can be startling.

The arm wrestling of the product management “market ask” captured in a Market Requirements Document followed by the team’s “development response” carefully defined in a Product Requirements Document is a case in point.  Weeks and months can go by during this ritual.  What are the development teams doing during this time?  Not at lot, in my experience.  Does development really “win” if they manage to push out the end date or reduce scope?  Does product management really “win” if they manage to squeeze in extra features or pull in the release date?  I’ve seen how the empowerment of the Product Owner and the establishment of the Product Backlog as the single high-level requirements and release scoping artifact helped to prevent some of these painful dysfunctions but I’ve never thought of them in terms of an imbalance of contract negotiation over customer collaboration.

Creating “Desire Lines” in Design

Larman and Vodde strongly advocate wikis as the preferred basis for the technical documentation the team develops.  They suggest a page for the overall product and pages for each Sprint. That worked for my teams as well, although there was always a certain (healthy) tension regarding what was appropriate for Sprint documentation and what should be living documentation at the product level.

They present the compelling idea of “desire lines”, which, in landscaping, refer to paths that develop naturally, as people use a given space.  Rather than guessing how people would use the area, they are observed using it and then the landscaping is designed around their actual usage.  Similarly in design, rather than trying to guess what the needs are, let them emerge and then refactor to support the emerging design tensions.  A lovely analogy, I thought, and one that will stay with me.

They suggest both a priori design in collaborative design workshops and design after the fact in System Architecture Documentation (SAD) workshops.  I like the acceptance that both approaches are going to be useful. The former stresses the need for team contribution to the design as the Product Backlog items are being developed, while the latter recognizes that team needs for technical documentation will emerge over time and so setting time aside to fine tune design documentation is both warranted and healthy.

The authors also address the question of whether and when to rewrite code – like Joel Spolsky, I personally favor refactoring to improve legacy code rather then re-engineering.  The authors present a compelling and complementary argument for refactoring and incremental improvement based on the value of instilling a notion of continual (rather than punctuated) improvement in the development teams.  They stress that the real problem isn’t the legacy code you’ve got but rather the legacy code you continue to write.  Encouraging the team to have a mindset of each check-in leaving the code base better than it was before – fixing the broken windows and taking out the trash – is a better solution to the problem of poor code quality than is carving off large sections of time for exclusively improvement-oriented work.

Acceptance Test Driven Development

The book’s worth it just for this material alone.  Larman and Vodde present a wonderful analysis of ATDD and characterize its place in the context of Scrum, including tying it into iterative requirements refinement, the Definition of Done and the Sprint Review.  I’d seen teams go in this direction driven largely by the desire to better engage testers in their teams throughout the Sprint and avoid “scrummerfall.” The authors take this further to show how ATDD, in which acceptance tests are defined and automated in advance of the development of the code that will pass those tests, does for teams in an iteration what Test Driven Development (TDD) does for individual developers in ten minutes.

They stress the need for the test automation to be a distributed responsibility shared by the whole team rather than one assigned to a specialist team.  I couldn’t agree more.  I’ve seen many attempts to establish test automation through the creation of a group apart from development and I can’t say that I’d call any of them particularly successful – at least not in the broad and encompassing sense that Larman and Vodde are envisioning in this book.

Continuous Integration at Scale

I was particularly glad to see treatment of Continuous Integration (CI) at scale. I’ve seen groups start with vanilla CI as it is described in the Extreme Programming literature and do well with it initially but then fail as their groups grew larger.

Essentially, Larman and Vodde propose a nested set of continuous integration builds, each larger one subsuming sets of smaller ones within it and each build defined for a particular concern.  Examples of these concerns include the verification of specific components and subsystems but also particular kinds of testing – including things like performance and stability testing.  One key aspect of this approach is that, at each level, you’re trading off the immediacy of feedback to the developer against the likelihood of the developer’s submission affecting quality at that level and the expense of the tests.

The Elusive Definition of Done

As noted earlier, Larman and Vodde,  suggest defining “done” as that subset of the work needed to release the product the team is currently capable of performing each and every Sprint – then use your retrospectives to challenge the team to bring more and more of the “undone” work into each Sprint.  The authors point out that there are really only two ways to extend the Definition of Done:  increase the cross-functionality of the team or increase the degree of automation the team uses.

In the meantime, however, while the Definition of Done leaves some work undone, Larman and Vodde suggest meeting that problem honestly, by doing things like defining an undone work team and pushing undone work onto the Product Backlog.  By being explicit about where the team currently stands against the goal of delivering potentially shippable product increments, this undone work can be better managed and the team’s insights can be directed to the problem of expanding that definition of done to get them closer to the goal.

For the Bookshelf of any Agile Team Member

The book isn’t without its faults.  It’s certainly long enough and some sections can be tedious (there’s a fifteen or so page section where different scenarios for story splitting are laboriously addressed – I get that this common complaint about stories needs to be put to rest but this feels like killing me with kindness :)).  The book’s organization lends itself more to use as a reference than as something you’d read cover to cover.   There are many repetitive sections that allow each chapter to stand on its own but are a bit hard to get through when you read it straight through.  But these are really just quibbles.  In all, “Practices for Scaling Lean  & Agile Development” is, just as its companion volume before it was, a tremendous book that belongs on the bookshelf of any agile coach, manager or team member.

I’d like to thank Anne Greenhaw and Selaine Henriksen for their help in developing this post.  Thanks also to Ryan Martens for inviting me to post here.

About the Author: Ed Willis has been a ScrumMaster, Product Owner, Scrum coach and trainer.  He is currently a developer in the telecommunications industry.

This is #3 in a Series on the Culture of Innovation with guest blogger Lee Devin.

Plan to do what you want. Prepare to do what you must.

Don’t get us wrong, we value planning: it’s important and highly creative work. But in the Culture of Innovation preparation means much more.  In a world that defines success as a result and failure as a step along the way , we plan regularly as we adjust to results, outside stimulus, and feedback.  Preparation marches us up the stairs faster and ensures that we’ll arrive someplace new and valuable.

Planning is an exercise for imagination and not spreadsheets.

In planning we figure out what we need to accomplish this task.  It‘s a process of creative thinking, dialogue, narrowing to convergence, healthy skepticism, and risk mitigation.  In planning we need to treat difficulties as a challenge; to resolve a creative tension between reality and what we want.  Teams brush away perceived limits as they press toward understanding by asking WHY?  Thinking in the 5 Why’s of Fishbone diagrams, these teams do not simply work with WHAT and HOW.  Once done and aligned, the plan becomes a communication of intent and result and NOT a goal or commitment.   Dependable results come from a focus on the limits to throughput, sources of failure, and lack of preparation.

In our experience with Agile teams, we see advanced Scrum teams begin to let go of some planning practices as their expertise grows. The benefits of pull-based planning and smooth flow delivery create new space for them in the market.  As a result of their growing confidence, they increase their ownership of their process, a key step on the way to a culture of innovation.   That culture creates, not just one off innovations, but an environment ready to take advantage of opportunities and happy accidents.   A big part of creating that environment comes from a focus on preparation.

Let’s consider preparation. Teams and managers must learn and practice a set of skills that taken together can help them create a culture of innovation. These skills often seem off the subject, not to the point, and therefore difficult for teams and managers to make time for. We think of preparation in three main categories: for collaboration and leadership; for comfort in ambiguity; and for daily productivity. In this brief introduction we won’t suggest a detailed program. Instead, we’ll outline an abstract of the culture, seen through the lens of preparation.

Collaboration and leadership

You can prepare for collaboration (innovative team work) and leadership (team direction and support) by learning and practicing release and concentration. Teams and their leaders need release from tension, as a way to increase available energy and flexibility; and release from inhibition and vanity for freedom, to include the work of others in their own and to regard the success of the team as their own success.

Take a look at athletes for good examples of release from tension; at actors in a play or movie for good examples of release from inhibition.

Watch Sharapova’s face as she looks up at the ball she’s about to whack; see the pitcher take a big breath and whoosh it out before he throws the ball. Look at a still photo of what the pitcher does to his arm in the delivery: it’s not hard to imagine what would happen to those muscles if they weren’t completely released, free of any kind of tension. Look at Paul Newman’s famous eyes blaze with rage (as Harry Manning, dumped in the river: Where the Money Is) or fear (Buffalo Bill astride a fractious horse: Buffalo Bill and the Indians).

We’ll use a story to illustrate what we mean by concentration. Once upon a time two students of Zen walked along the lake shore. They spoke as follows:

First Student: “I have the world’s most amazing Master.”
Second Student: “Have you?”

First Student: “He performs miraculous deeds. The other day he walked right out on this lake and spoke to us, standing on the surface of the water. Then he walked back, and his shoes weren’t even damp.”
Second Student: “That’s certainly amazing. I congratulate you. My master, however, can do something much more important and amazing.”

First Student: “No way.”
Second Student: “Yes way. My master can do one thing at a time.”

Who among us can do one thing at a time?

As you plan your week next Monday, think about these questions.

• What is the #1 Thing you have to get done right this week? Be clear about that to yourself and with your team and put your best time and focus on this one item.
• What preparation or practice can you do to release tensions with regards to this item?
• Who can you collaborate with to make this an outstanding result?
• What can you do to celebrate the results of this effort?

What might you do to prepare to execute these choices? What kinds of practice might you build into your daily, weekly, monthly, routine?

Comfort in ambiguity

Accident, serendipity, new things. Innovation confronts the team with all of these sources of ambiguity. What’s gonna happen? What should I do? What on earth is this thing? How do we know when it’s complete?

How does preparation contribute to comfort in ambiguity? It gives us grounds for confidence in our ability to manage the new, the surprising, the unpredicted. We don’t need to dread the uncertainty of innovation because we know that we can make good use of whatever comes up.

Teams and managers who do innovation find ways to live with uncertainty about the outcomes of their work. They know that outcomes will be unexpected and surprising. If they could anticipate them, how new could they be? Preparation will involve getting free of the reflexive invocation of the past: “That isn’t how we do things here”; and embracing the uncertain future: “Let’s see what happens when we do this.”

Preparation will sometimes replace planning.

Of course we plan, so that we can do what we need to do. We plan to have the materials we need, space to work in, the right people on the team, to make an efficient schedule. Planning creates sequential progress toward a known goal. Preparation, on the other hand, aims at collaborative iteration toward an emergent outcome. No one can predict the results of a true collaboration. We prepare to cope with whatever happens. In a culture of innovation, whatever happens is likely to be new. It will elude any kind of sequential progress toward a known goal. When an outcome doesn’t seem to have any immediate value, we recognize that nothing is lost: we set it aside (Might come in handy some day.) and press on.

The notion of emergent design conditions any serious innovation.  At Rally Software, we do a number of things in the context of Agile software development to keep from planning too much and to hold space for reaction to ambiguity.  First, we acknowledge multiple levels of planning with less precision as the time frame goes out.  Second, we insert free time into our schedule in the form of slack and programmed innovation time.  Third, we work “sets” of designs through a narrowing process to keep from choosing the design before we learn.  Finally, we do not plan until after we have closed the last cycle: We check the results of that last cycle and consciously review our goals.  We “Plan to get lucky” and provide room for that to affect our next cycle.

We took a young engineer to visit an acting class at People’s Light, the theatre we know best. A bunch of teenagers were practicing improvisation. One sat on a bench in the park. Another passed by, stopped to talk. A story began to develop. Suddenly from the class a third jumped up and walked into the park, joining the two. This newcomer brought an entirely new slant to the story. After a moment the first actor remembered an appointment and left the other two. Someone else from the class joined in. And so on. The story grew, got elaborate, got simple, got turned inside out: the kids never repeated themselves, never stopped. No one ever refused the new material offered by an other. The engineer turned to us and whispered: “This is exactly what my guys need to learn how to do.”

This kind of practice fairly closely resembles the desired skills. Engineers like to look for an answer in the back of the book; they need practice in making up answers out of the available material. The kind of preparation we’ll call exercise strays from the skills it prepares for; it’s off subject, away from the actual work. Athletes exemplify this kind of preparation. “The champ,” goes the saying, “is always in the gym.” Larry Byrd was famous for staying in the gym after practice. Why? To shoot 100 free throws. To build a reflexive confidence that supports the hectic innovations of the game. What’s more, the champ has decided, has made the choice, to like being in the gym; how could he do the work otherwise?

As you plan your week next Monday, think about these ways of practicing or preparing for emergent innovations:

• Schedule some creative time into your schedule to spend in a creative place and time.
• Step back from your #1 item for the week and ask yourself a question about its value: What other things could I do to deliver even more of this value?
• Find one example of yourself closing down to new solutions based on the concept that “This is the way we always do it.” Can you release that constraint?
• Ask yourself: What is the most important thing I have to do this month or quarter?  Not urgent. Important. Do I have enough time, support, and space to do this right?  Try removing less important or merely urgent things from your calendar to make room for this.

Daily productivity

In a culture of innovation, everyone chooses a professional obligation to work happily, enthusiastically and at maximum energy.

Artists and athletes again serve as models. Neither group can do what they do unless they’re totally fired up. High morale and physical readiness are basic conditions of their work and they must learn how to create and maintain them, no matter what. An actor arrives at the theatre well before the half hour call (On time is already late.), and begins the day’s work with an extensive warm up. Vocal exercises, calisthenics, stretches, lines; actors have routines they follow religiously.

An actor we know told us this story. He used the 30 minute drive to the theatre as his time for vocal warm up. One night, distracted by some domestic emergency, he only got through part of his routine by the time he arrived at the theatre. In rehearsal he had discovered a way of saying one of the lines in the 2^nd act that every one liked a lot: his voice got deep and sexy, very nice moment. On this night the performance went very well, in spite of the truncated warm up. Until that deep sexy part. He said that line exactly as he had done dozens of times before. But instead of deep sexiness, what came out of his mouth was tired little squeakiness. The audience were too embarrassed even to laugh. You can bet that actor never missed another warm up.

In software development, this is akin to doing some manual work outside the scope of your automated build, deploy, test cycle.  It can seem quicker to do a simple, one-off integration or system test outside that environment, but unintended consequences always catch-up .  In our experience, cutting the preparation corners usually costs 10X more in the whole lifecycle than it saves in the short-term. Beyond the interrupts of customer-impacting defects, the team loses a bit of the pride and belief necessary for the Culture of Innovation

Team work demands a no less total performance than acting or tennis playing. It needs, but rarely gets, the preparation of a warm-up. A basketball team combines individual warm ups (stretches, shooting around) with group work (lay up and passing drills). Why should knowledge work be any different? Imagine the energy available if your morning stand up included a vigorous warm up led by a different person each day. Jump back!

As teams and organizations reach an Innovate level of Agile adoption or Ri , they take ownership of their process and environment.  Their improved throughput, collaboration, and preparation have brought them to a place where many of the vanilla iteration, planning, and estimating practices of Scrum and XP stop serving them.  These structures helped the incremental transition down a path of increasing agility, but in a Culture of Innovation, where smooth, continuous flow of one thing at a time is the goal, the focus moves from planning to preparation.

Next up in our series – Options, Fall-back and Design Sets

About the Authors: Ryan Martens is a goat cheese maker,  founding board member of the Entrepreneurs Foundation of Colorado, and Founder and CTO at Rally Software Development.

Lee Devin is a dramaturg at the People’s Light and Theatre, a Senior Research Scholar at Swarthmore College and a senior consultant with the Cutter Consortium. These activities often interfere with his fishing, and cause him to neglect his grandchildren.

Subscribe today to get free updates by email or RSS.

Looking for the perfect trend article to pass on to your executives about Agile development? Check out Dan Woods‘ well-researched, executive-level article “Why Lean and Agile Go Together” on Forbes.com Jargon Spy.

Dan describes how the Agile impact in software is similar to the Lean impact in manufacturing of the 90’s.  I initially struggled with the analogy until I realized he is not saying that software development is like manufacturing. He is saying that the concepts and techniques applied in Lean manufacturing are coming to large-scale software development.

I completely agree and believe the model of how we build software is changing just like Dan describes.  Using the model from the Fifth Discipline Fieldbook, the change looks like this:

Lean Software Concept

That was a great decade – I see much of the last decade of improvement coming from a real effort to take responsibility for how we perform our craft.  The ground up work in Agile, Scrum, XP and Lean has increased velocity, increased quality and increased the quality of life for many in our industry.  I am personally in a much better working environment; how about you?

So what’s ahead for 2010?- With Agile software development hitting the mainstream, the guiding ideas on how to design and deploy software is changing for most software driven organizations.  As more teams and organization make the transition to the Agile/Lean triangle, I see us increasing our investments in skills training, teaching, experimenting and innovating with an increasing number of new tools, techniques and methods.  As a result,  I see this decade moving us past the Lean models of Toyota to something uniquely our own.  I am really excited about the improvement this decade could bring to our industry and all the industries that we impact with our software.

About the Author: Ryan Martens is a goat cheese maker,  founding board member of the Entrepreneurs Foundation of Colorado, and Founder and CTO at Rally Software Development. Subscribe today to get free updates by email or RSS.

Over the last several months, I led four breakout sessions on Agile quality at our Agile Success Tour and in the process I heard these troubling comments:

“We don’t have enough time to test, because the developers give us working builds way too late in the cycle.”

“I wish our quality was better, but it’s out of my control to change anything.”

“Poor quality must be our fault as testers, since we are the ones responsible for testing.”

“We should test more, or we should develop slower, or we should specify our requirements better – that should improve quality.”

Agile development advocates that the whole team owns quality, but clearly many teams face barriers to accomplishing that goal. I’ve been thinking about these comments a lot lately, and I think the problem is that basic and predictable human responses are getting in the way of achieving higher quality.

You can certainly improve quality through changes in process (and many of those changes are at the center of Agile adoption), but more often when people ask me process questions, I hear teams struggling with the human aspects of those process improvements. By paying attention to the ways everyone naturally avoids responsibility, you will be able to move past the barriers that contribute to poor quality.

The best model I’ve seen for how to do this is Christopher Avery’s Responsibility Process. The basis of this model is that when problems arise, every individual goes through a very predictable series of thoughts that allow him to cope with what just happened, but also block him from moving into a mindset of learning and improving.

It starts with blaming someone or something. When you move past blame, then you justify the event with a story you tell yourself. Once you move past that story, then you turn the blame to yourself and feel shame. Acting on that shames often leads to feeling a sense of obligation. If the shame and obligation become too unbearable, then you will quit or detach to escape from those feelings.

All of these thoughts prevent you from taking responsibility and ultimately moving into a mindset of learning and acting. Sound familiar?

This same pattern subtly plays itself out every day for you and your team around you, especially with issues of quality. According to the Responsibility Process, there are three keys to taking responsibility: (1) intention, (2) awareness and (3) confront.

To learn more, I encourage you to join me and Christopher Avery for our January 12th On Demand webcast, The Best Kept Secret in Agile Quality, where we will talk about :

• the relationship between quality and ownership
• the difference between accountability and ownership, and how this difference affects quality
• how the Responsibility Process can be used as a framework to continuously improve quality through a focus on people and interactions
• how to apply the Responsibility Process to your development team

I’m really excited about what we’ve put together for this and would love to see all of you attend.[Reserve your spot now by CLICKING HERE]

About the Author: Zach Nies is a golfer, expert on Agile quality and VP of Products at Rally Software Development. Subscribe today to get free updates by email or RSS.

Recently, I wrote the post “Escalation is Killing Agile – Can We Please Stop It?”  My passion around escalation brought 40+ in-depth comments.  With my travels and lack of internet access, I had been unable to sit down, sift through, and absorb all the various perspectives.  Until now.

I’m offering this follow-up post as a means to provide an overall response to all these great comments. I want to add some further background on the “escalation” topic and some more of my personal conviction around it. Specifically, I’d like to provide some insights into delayed feedback, the need for conflict, and how to “show up”, all without escalation.

In one part of the comment stream, I heard and felt the call for an effort to get to the root cause of such deep-rooted assumptions.

According to the Systems Thinking Toolbox from Pegasus Communications, to break an Escalation structure, ask the following questions:

1. What is the relative measure that pits one party against the other and can you change it?
2. What are the significant delays in the system that may distort the true nature of the threat?
3. What are the deep-rooted assumptions that lie beneath the actions taken in response to the threat?

So, in our system of sharing information in the Agile community, we have to ask, “Are we setting up a dynamic that pits us against one another?” If this is true, then we have to ask, “How can this be addressed and still ensure that we share insights?”

Guided by Systems Thinking, that means we need to check in with: what is distorting our communications and what might our deep-rooted assumptions be that would have us act as we do?

Here is an example:

I created a delay in feedback by my not responding to posted comments. I believe that created assumptions around what I may or may not have intended in the post.  I think some individuals thought I had written the post pointed specifically at them. Faster feedback would have helped quell that assumption.

I was writing about, and continue to write about, the Systems Thinking Escalation archetype and how I see it in our community. I was and am looking at a dynamic not at an individual. Escalation is NEVER about an individual; it is about the system in which blame is occurring and allowed to continue. I am fearful that blame and the win/lose game are in our system and that is what I do not like and I want to address.

Some of the comments to my post seemed to indicate that I was anti-conflict. Far from it! In studying the inner workings of high-performing teams, I have often referred to Sam Kaner’s model for participatory decision-making. Conflict is a must.

In this model, Kaner insists that, to get to high performance we must bring forth conflict to discover the best informed decisions. Divergent ideas must be invited. Divergent voices must be heard. Divergence must be allowed to just be. That is, don’t just jump to conclusions. With enough time and patience around divergence, we can then move toward informed convergence.

Conflict in this context is dialogue. It seeks insights. It invites greater and greater participation. I also want to emphasize that in this context of dialogue and non-escalation, our purpose is to engage in forward thinking. We let go and we look forward. And as we look forward, we let go.

So, as a member of the Agile community, my interest in expending energy in discussions is to seek insights, encourage divergence, and discover convergence as it emerges. All of these practices help and encourage me to create more and more forward thinking. If this is not occurring in our community, then we are not getting enough for the energies we expend.  If in our community we really “must win”  in order to “be heard”, we are stuck in an “Escalation” archetype. And, that means we are all trapped on an up escalator to nowhere.

What could any individual do to break an escalation pattern in a system? Create energy around your insights and share them without a need to apply win/lose stakes. Stop expending energy to refute others.

Here is a simple formula for bringing your viewpoint to bear without escalation:

1. Show up. (Be willing to be engaged.)
2. Find out what has meaning to you. (Be willing to be honest about your perspective.)
3. Tell your truth. (Be articulate about your insight without attacking or assigning blame.)
4. Let go of what happens. (Be courageous enough to allow others to agree or disagree.)

I believe this formula provides guidance on how to remain forward thinking, remain open-minded, and remain engaged.

I have some more mental models I want to offer here. But they will wait for another post.

Thank you in advance for your considerations, insights, and comments.

About the Author: Jean Tabaka is a wine enthusiast, author and Agile Fellow at Rally Software Development. Subscribe today to get free updates by email or RSS.

Sarah: “Walter, I just want to check in with you following the team demonstration and see how you are doing with the new Agile approach”

Walter: “Sarah the results were thrilling for the customer and the team.  Everyone seemed engaged and the dialogue was very healthy. I see it, but it does not make sense.  We are moving features through the team faster, but I had to do this with dedicated resources.  I am sure this is costing me more.”
Sarah: “Walter, this is totally normal.  You are seeing the difference between single tasking and multi-tasking as well as optimizing the whole versus a certain roles or steps.  We can do a little model to show you that this Agile approach is better, faster and cheaper.”

Walter: “Well that would be very good because it is very counter-intuitive to me and all my management tools and tricks seem to run at odds with this approach.”
Sarah: “Great observation, Walter!  As you told me earlier, you tend to focus on fully allocating engineers.  In Agile we focus on increasing the throughput of value by optimizing the whole flow through all roles.”

Walter: “Right now I just want to stay out of the way, because I do not know how to help.”
Sarah: “Walter, let’s spend some time to get you comfortable with the impacts of batching, queuing and task switching.  Once you see the math, we can talk about how to serve the team by proactively clearing impediments to smooth flow and taking some savings to invest more in them.  The team will love you for this commitment.”

What benefits will you see when teams are in Flow?

As we described in “What so Great about Flow?“, we think of attaining Flow as the first step in our recommended approach to enterprise Agile adoption.  For this step, we focus groups on adopting agile by the book by team; thus the impact from Flow is only incremental and isolated to the specific team.  However, this does not mean the results of this effort are not noticeable and infectious.

Foremost, groups that attain Flow make changes on the way to becoming a team.  When given space and dedicated members and short time box, new agile teams take three key steps:

1. First the collection of individual contributors comes together to plan a limited portion of their own work. 
   
2. Second they set norms and limits that lead to common commitment to deliver working, tested increment. 
   
3. Finally, they open their increment up to feedback and open themselves and their process up to feedback, introspection and adjustments for continuous improvement. 
   

The benefits seen from doing this are many:

• A team is forming and that makes work more fun and easier for them to manage
• Work-in-process is getting reduced by focusing the work on a small increment of functionality
• Testing is coming forward in the process and quality is beginning to become a team issue
• Hand-offs and delays are declining by dedicating a cross-functional team to working on one thing at a time
• Feedback loops are shortening in the demonstration and review process helping to prioritize the next step

Typically these benefits take one to three iterations to manifest themselves and translate into incremental productivity increases of 15%, 25% faster cycles and a reduction in downstream defects (See the Agile Impact report for more details of business improvements from Agile).  Most of these teams are reporting the benefits of flow.  Teams in Flow settle into a predicable pattern of delivery.  They are not perfect, but you can count on them and estimate based on their historical velocity. We look for these metrics to know if a team is in the state of Flow including:

• a rate of 90% or greater of Story Acceptance per Iteration on an iteration over iteration basis
• an average of zero net, new Defects added to Defect list across iterations
• A reduction in 1/2 of the amount of stories that are in-process across iterations
• A continuous build success rate of 90% or greater for the team’s isolated code base

In general, they are starting to do things in a more lean and less wasteful way.  By working together on a small batch of work with a clear definition of done, it allows the team to focus and reduce the task switching.  To understand why multi-tasking and task switching do not work, you should read this post from Mark McGuinness at Lateral Action or check out Tom Demarco’s book “Slack” and specifically pages 16-21.

However, the most important benefit is the positive feedback from the team members with regard to forming a team and working in this intuitive and social approach.   It is this positive feeling that propels teams to move past Flow and into Pull.  Congratulations if you have the continuous improvement snowball beginning to roll downhill at your organization.  Building the trust and capabilities of the team is the most effective path to more value delivery.  (See “The Scrum Picture is wrong” and the comment/link about the Lean Journey from Jason Yip.)

If your team does not find this benefit, it is clear that there is a fundamental issue that needs to be resolved.   This is a great time to bring in an outside coach to help the team retrospect and give them space to form.  In these situations, it is easy overlook the fact that teams need to go through the process of of forming, norming, storming as they reach toward performing.  Remember this is a journey and you can not go there without building the team.

About the Author: Ryan Martens is a fly-fisherman, founding board member of the Entrepreneurs Foundation of Colorado, and Founder and CTO at Rally Software Development. Subscribe today to get free updates by email or RSS.

Introducing Rally's Engineering Blog

As some of you have already begun to notice, the Rally Engineering team is now in full effect with their very own Blog.

This blog provides a great opportunity to (1) gain further insight into Rally’s development environment, (2) ask questions and interact with a high performing Agile engineering team and (3) learn tip’s and tricks from the team’s experience working in a wide variety of technologies.

I have read all their posts and there is something in there for all the rolls and levels of experience on agile teams.  It is unique because of the open authoring model for all members of a fast growing set of agile teams.

With 15 posts published, and many more regular contributions on the way, I invite you to join me as a new subscriber to Rally’s Engineering Blog (click here to subscribe and here to visit the blog)

About the Author: Ryan Martens is a fly-fisherman, founding board member of Entrepreneurs Foundation of Colorado, and Founder and CTO at Rally Software Development.

In the last couple of months, IBM (via Scott Ambler) has blogged, hosted webinars and talked to the media about an Agile process maturity model (APMM).  I am sure this will hit a new height today with the start of the Rational Users Conference and the release of the e-book.  I have been asked to comment on this work by a number of press and analysts.  Since my perspective will be published shortly, I thought I would go on record.

IBM splashes its way into Agile development

As the Agile market grows and takes hold in truly mainstream audiences, everyone is looking for easy, step-by-step guides to smooth Agile adoption. IBM is proposing one option under the name “Agile Process Maturity Model.”  I think it is a nice marketing strategy for selling IBM’s Rational Team Concert products to companies that want to adopt the Jazz platform, copying their 90’s success with RUP.  But I don’t believe it is actually an Agile process maturity model, and further, Agile doesn’t need one.

IBM’s proposal for an Agile Process Maturity Model

Scott says, “The goal of the APMM is to help categorize and understand agile processes, not to rate your adoption level (the CMMI Defined approach can address that need).”

He continues, “Unfortunately, the term ‘maturity’ is a loaded one within the software process realm, not the least because of the Software Engineering Institute (SEI)’s Capability Maturity Model Integration (CMMI). A lot of good work has been done to show that agile and CMMI can be applied together, and I look forward to seeing that strategy come to fruition. However, where the goal of the CMMI is to provide a framework for software process improvement the goal of the APMM is much more modest – it merely strives to define a framework which can be used to put the myriad agile processes into context. In short, the APMM and CMMI are orthogonal to one another…” 

I am confused by the title and the orthogonality, so let’s peel the onion on this one.

What is a process maturity model?

The origins of process maturity models come from the manufacturing industry, but the software version was created in the 1990’s by Carnegie Mellon’s Software Engineering Institute (SEI). It is well known as three Capability Maturity Models – CMM for software, CMM for people and CMMI.

If you read Watts Humphrey’s work, you will see this is a management framework that measures the level of discipline of  your organization. In essence, organizations that are certified Level 5 have implemented continuous improvement and have the disciplines and practices in place to effectively manage large complex projects with effective controls.  The framework does not say anything about how the software should be developed.  I make no claims to being a CMMI expert, but you can read a few of the “CMMI agile” search results to see how they work together.

Though not the stated intent of the framework, it has been closely associated with waterfall development and used to defend heavy process approaches. I disagree with that belief and prescribe to Jim Collin’s model of blending  discipline and agility together to move from good to great. I think the CMM framework is a great checklist for organizations measuring their level of discipline… and we don’t need another one.

The 3 levels of IBM’s APMM

Scott’s APMM post describes three levels:

1. Core Agile Development
2. Disciplined Agile Delivery
3. Agility at Scale

I struggle to see how these three labels or the supporting details provide a framework that helps categorize and understand Agile processes.  These titles grow in scope and scale, but do not speak to increasing agility.  (Faster cycles, less waste, less work-in-progress, more value.) When you look deeper, it smells of a cookbook of when to apply processes and tools from IBM.  That is fine, and I agree customers need this. However, I would argue this should not be used to guide companies in their Agile adoption.

By mashing size, scope, increased agility and disciplines on one scale, the APMM does none well.  In addition, it will lead to amateur implementations where continuous improvement mentality never sets in. Where adopting scaling software agility is thought to be just a transition from A to B.  To me that thinking is very limiting and will lead to gains followed by declines.

IBM in the Agile space – all good

I am thrilled with IBM’s entrance into the Agile space with their Jazz platform and related products – really. When I was at BEA/Weblogic, I can tell you that we did not have a market for Java Application servers until the industry’s gorillas, IBM and Oracle, entered for real.

During this year, Jazz and its supporting products will actually manage the workflow of projects in a way that works with Agile development instead of against it. This is huge change from the ALM 1.0 point products that actually reinforced the silos and phased approaches that created queues, caused hand-off delays and kept quality as an after-thought.

The IBM gorilla in the software development space will make the Agile sea rise and help break down major organizational barriers to bringing the benefits of Agile practices and tools to the mainstream development community.

There is no cookbook for adopting Agile

I believe that enterprises need an adoption model that helps them balance discipline and agility in an incremental fashion that creates incremental success and fuels continued investment and improvement. The enterprise Agile adoption model Jean Tabaka and I have synthesized is based on Lean concepts and is rooted in Deming’s work, just like Watts Humphrey and CMM. What Jean and I present in our whitepaper titled “Leaning IT: Moving to Program Pull” is how to move up the adoption curve of Agile by learning quickly,  maturing before scaling and working incrementally. 

This transition planning framework focuses on attaining benefits of agility by moving to lean states of flow, pull and ultimately the perfection of continuous innovation before adding the disciplines necessary to scale.  It discusses the general steps, but does not prescribe a change approach to organizational, technological, process or strategy transitions.   Your approach is dependent upon your corporate and technological situation.  We recommend an iterative approach, facilitated by experienced coaches and peer support.

What does the Agile community need from IBM?

Mary and Tom Poppendieck’s new book called “Leading Lean Software Development” (out soon) has tremendously strong stories, models and insights from bringing lean concepts to software. In chapter six,  they highlight a success story from working on lean with Sue McKinney at IBM.  This success story, along with the Eclipse success stories, are critical for IBM to be telling the market regarding the impact of Agile and Lean in the large inside IBM.

In 2008 and four years after IBM started developing Jazz, we talked to IBM about integrating Rally’s award winning Agile Lifecycle Management solution onto IBM’s collaborative development platform, called Jazz Foundation.  They said no, you have to wait until the GA release of Jazz. This would be a similar integration that we provide to Microsoft Team Foundation Server for managing code check-in’s and automated builds with a slick interface into the IDE.  I am hopeful that IBM will GA the 1.0 version of the Jazz Foundation for partners to actually integrate with.  They are saying all the right things on the Jazz.net site, here is hoping for some positive news at this week’s Rational User’s Conference.  Finally, I am very excited about IBM’s global dialogue on Building a Smarter Planet. IBM needs to do a better job telling the story about how software agility is a critical component to building that sustainable planet. (It was nice to see the smarter planet splash at the front of the ebook, but it still represents a huge missed opportunity.)

What do you think? Does Agile need its own process maturity model?

Again, I was surprised by the title of IBM’s Agile Process Maturity Model and hope they will consider changing it before more marketing dollars are spent.  It does not improve on the CMM, nor in my opinion does it help CMM certified companies adopt Agile.  I assume IBM has a huge role to play in helping those CMM certified companies add agility and innovation to their highly disciplined organizations. If they don’t, we are glad to help today.

About the Author: Ryan Martens is a happy father,  founding board member of the Entrepreneurs Foundation of Colorado, and Founder and CTO at Rally Software Development. Subscribe today to get free updates by email or RSS.

#5 in our list of the Top 10 characteristics of an Agile organization is about the importance of practicing both Bottom-up and Top-down decision making.

In this 4 minute video, I talk about how successful Agile organizations embrace a notion of the ‘knowledge-creating company.’ In Agile, knowledge-creation can use “5 Levels of Planning” to ensure they are engaging in this whole organization practice. In sum, the highest level of planning, the vision, feeds and is fed by all subsequent levels, down to the lowest level of planning, the detailed daily work.

Watch my video for more about why I truly believe in both Bottom-up and Top-down decision making as a key success factor.

See our previous coverage of #10 Work/Life Balance, #9 Being a Servant and Leader, #8  Sustainable and Successful, #7 Contributing to the Community and the Company and #6 Collaborative and Smart. Stay tuned for #4 in our series, Personal Flexibility and Rhythm.

In an earlier post, I asked an Agile test question, “Are PMOs Obsolete in Agile?”

While the answer MAY be “Yes!”, I posited that there may be a way to save our PMOs from going the way of the now extinct dodo bird.

8 Ways To Re-Tool a PMO in an Agile Environment

1. Have the PMO involved in helping rollout Agile on several pilot projects

   This experience forms the roots for how the Agile PMO will engage throughout the organization with other project teams.

   Besides ensuring that Agile training occurs, the Agile PMO can become the coaches of coaches, training the ScrumMasters of each team. They can also be the Agile voice into the business to ensure the Product representatives are fully bought into their role in Agile.

2. Learn from Lean: start a Kaizen and Gemba mentality now

   The PMO should be the shepherds of continuous improvement so that the Agile adoption is not only sustained; it is vibrantly organic, every improving. This means moving beyond team-level iteration-focused retrospectives.

   Kaizen events create cross-team learning. This learning exposes team practices that can be useful at an organizational level. In such cases, the team practices are elevated to the organizational “Standard of Work” (see later item). Moreover, the Kaizen event clarifies when a practice does not serve the entire organization and can remain a team practice, supported by team agreements and adhered to by the team.

   In addition, in between Kaizen events, apply “Gemba”. An Agile PMO learns about the organization and spreads knowledge through the organization by being involved in projects. That means going to see, or “Gemba” according to Lean practices.

   Being engaged with teams rather than dictating from afar may be a dramatic shift in PMO behavior. And, it is pivotal to teams’ respecting the role of the PMO in creating a successful Agile adoption.

3. Change your definition of “Standard of Work”

   This is not a set of standards defined by and enforced by the PMO for project conformance.

   In Agile, as guided by Lean practices, a standard of work is a statement of what is currently useful, how things are currently done. It provides a team a backdrop from which to perform continuous improvement through Kaizen events such as retrospectives.

   (In my previous post on the PMO, Brendan Flynn has a wonderful comment about how he and his PMO engage teams at Shoplocal in doing this work.)

4. Facilitate and be a Servant Leader

   Don’t control.

   The Agile PMO, rather than controlling and patrolling, engages in facilitating teams in continuous learning. This PMO encourages empowered teams and amplifies the learning that arises when such teams are truly empowered.

   A great way to provide this facilitation and to engender cross-team involvement is to help organize and facilitate program-level and multi-program release planning meetings and product councils. The Agile PMO formulates a rhythm and a continuously improving agenda to help these meetings deliver the greatest benefit across teams.

   Along with being facilitative, Agile PMOs act as Servant Leaders. They serve their organizations by finding out what they need and getting it for them.

   They lead by serving. They remove impediments. They measure what is a problem for teams so that they can serve teams be removing the problem. They don’t look for status from teams; they look for how they can serve teams.

   So, for an Agile PMO, leading the Agile rollout means finding out what teams need and delivering that. It is about creating collaborative plans, not collecting statuses. For more about Servant Leaders in Agile, check out the first section of my book Collaboration Explained.

5. Provide guidance on regulatory adherence

   In organizations that must adhere to Sarbanes-Oxley, FDA, HIPAA, or other regulatory guidelines, the Agile PMO can represent these “red” (immutable) stories in each team’s Product Backlog.

   The PMO governance work can be the guide to each Product Owner on how to deliver the documents or features that help the entire organization remain in compliance with these regulatory constraints. Governance is now a service to project teams versus being a control mechanism.

6. Seek guidance from Lean to use Flow, Pull, and Innovate as guidelines for organizational-level Agile adoption

   Much of the guidance here has a hint of Lean about it. That is because PMOs work at an organizational level. Moving Agile beyond one or more pilot teams requires guidance for maturing and scaling.

   I recommend applying a 5-step approach which is based on Full, Pull and Innovate principles from Lean Thinking.

   Our Rally whitepaper about Applying Lean Pull principle for Program level Agile is a great guide for the Agile PMO working with programs of teams.

7. Guide non-IT/Engineering organizations

   Once Development group is working with the business to deliver prioritized, valued items, the Agile PMO can bring in other ‘back office” functions to help ensure a growing Agile adoption.

   Finance, HR, infrastructure, the executive team, and others must eventually all accept the Agile approach for true organizational change and value.

   The Agile PMO has an opportunity to be the mentors and guides for this growth and maturation.

8. Foster useful metrics

   Let go of that Waterfall security blanket around what are useful project or program metrics.

   Determine which metrics to track that directly help organizations concentrate on value delivery.

   Whatever does not deliver value is waste. It may be temporarily necessary waste. Or it may be a pernicious, embedded waste that has been assumed to be necessary.

   An Agile PMO ferrets out these “false friend” metrics and works with teams to determine metrics around value: test coverage, team velocity, agile adoption assessment, defect counts, definition of done.

   Look in the Appendix of our whitepaper A CIO’s Playbook for Adopting Scrum for more ideas on useful metrics in Agile adoption.

About the Author: Jean Tabaka is a wine enthusiast, author and Agile Fellow at Rally Software Development. Subscribe today to get free updates by email or RSS.