Agile Development Blog

Here’s something obvious I’ve learned the hard way: delivering “potentially shippable product increments” each and every Sprint isn’t easy.  Essentially you’re trying to take all the disparate activities that occur throughout the waterfall process, focus them on just the little product increment’s functionality and then jam them into a teensy little Sprint.  Hard to do and definitely pretty unlikely to get accomplished right out of the chute. The authors of “Practices for Scaling Lean & Agile Development”, Craig Larman and Bas Vodde, forgive you in advance for not getting this done straight away. In fact they suggest that this is more a goal the team will approach over time than a rule they put in place on day one.

The authors propose something very simple but very insightful:

• Sketch out all the things you need to do to get your product out the door.
• Define “done” as that subset of those the team is currently capable of performing every Sprint.
• Use your retrospectives to challenge the team to bring more and more of the “undone” work into each Sprint.

This has already changed the way I think about retrospectives. For other nuggets from Larman and Vodde’s book, read on.

Overall a Must-Read for Agile Development Leaders

I was blown away by “Scaling Lean & Agile Development”, as you can see from my bullish review on O’Reilly.  Some time has passed since then but I still feel that it’s one of the most important development books I’ve read.  That book alluded to the companion volume, “Practices for Scaling Lean & Agile Development”, and as you can imagine I awaited its publication eagerly.  It came out in February – I’ve worked my way through it now.   It’s most definitely a worthy successor.

The first book presents theoretical and philosophical underpinnings for agile and lean development. The second book presents a survey of practices relevant to all aspects of the process of developing software at scale, presented by two guys who bring a wealth of knowledge and experience to the table.

Continual Investment in Requirements

Larman and Vodde present practices relevant to a continual investment in requirements throughout the product development process. This is done both up-front, in seeding the product backlog, and iteratively, in refining requirements to support upcoming Sprints.

I love the emphasis they place on whole-team involvement in the initial Product Backlog development effort – even for projects with large teams.  Too often I’ve seen this be the provenance of Product Owners, working in near isolation, which does little to get the project off on a good footing.  The first the team sees of the requirements is the Product Backlog itself, having been involved in none of the  discussions and thought that went into it.

The notion of requirements areas, which was introduced in the first book, is leveraged here to help parallelize the initial requirements development work.  Concurrent sizing and value estimation workshops keep the requirements work rooted in reality.  They spend some time on the problem of bootstrapping a consistent sizing process in a large scale team. The use of cross-team estimation sessions result in the development of a canonical set of sized stories used as a basis for subsequent sizing at the team level.

In Favor of Forward-Looking Requirements Refinement

The discussion of forward-looking requirements refinement really resonated with me. I’ve found in my own practice that peeking ahead at the Product Backlog items upcoming in the next Sprint and then spending time together working through them to understand what they really mean – before we get into the Sprint planning session – goes a long way towards supporting a predictable iterative process.

It also puts the requirements elaboration just barely outside the Sprint that the work is planned for.  This separates coming to an understanding about what we want from the stress of figuring out how to fit it into a Sprint, which makes for more open and enjoyable requirements development.  The authors suggest the use of examples and Acceptance Test Driven Development (ATTD) to more precisely capture the intended behaviors of the stories in a way that realizes both conversation and confirmation of the “three Cs” . Smart.

The authors show how traditional approaches to project scoping and commitment (where separate product management and development organizations act essentially as competitors) are very much analogous to the contract negotiation that the Agile Manifesto cautions us against.  I’m surprised to admit this never occurred to me – I always read that part of the manifesto pretty literally. But it’s inarguably true that the wrangling between these two groups over project scope and timelines that happens in many organizations is a form of contract negotiation, and the waste it drives can be startling.

The arm wrestling of the product management “market ask” captured in a Market Requirements Document followed by the team’s “development response” carefully defined in a Product Requirements Document is a case in point.  Weeks and months can go by during this ritual.  What are the development teams doing during this time?  Not at lot, in my experience.  Does development really “win” if they manage to push out the end date or reduce scope?  Does product management really “win” if they manage to squeeze in extra features or pull in the release date?  I’ve seen how the empowerment of the Product Owner and the establishment of the Product Backlog as the single high-level requirements and release scoping artifact helped to prevent some of these painful dysfunctions but I’ve never thought of them in terms of an imbalance of contract negotiation over customer collaboration.

Creating “Desire Lines” in Design

Larman and Vodde strongly advocate wikis as the preferred basis for the technical documentation the team develops.  They suggest a page for the overall product and pages for each Sprint. That worked for my teams as well, although there was always a certain (healthy) tension regarding what was appropriate for Sprint documentation and what should be living documentation at the product level.

They present the compelling idea of “desire lines”, which, in landscaping, refer to paths that develop naturally, as people use a given space.  Rather than guessing how people would use the area, they are observed using it and then the landscaping is designed around their actual usage.  Similarly in design, rather than trying to guess what the needs are, let them emerge and then refactor to support the emerging design tensions.  A lovely analogy, I thought, and one that will stay with me.

They suggest both a priori design in collaborative design workshops and design after the fact in System Architecture Documentation (SAD) workshops.  I like the acceptance that both approaches are going to be useful. The former stresses the need for team contribution to the design as the Product Backlog items are being developed, while the latter recognizes that team needs for technical documentation will emerge over time and so setting time aside to fine tune design documentation is both warranted and healthy.

The authors also address the question of whether and when to rewrite code – like Joel Spolsky, I personally favor refactoring to improve legacy code rather then re-engineering.  The authors present a compelling and complementary argument for refactoring and incremental improvement based on the value of instilling a notion of continual (rather than punctuated) improvement in the development teams.  They stress that the real problem isn’t the legacy code you’ve got but rather the legacy code you continue to write.  Encouraging the team to have a mindset of each check-in leaving the code base better than it was before – fixing the broken windows and taking out the trash – is a better solution to the problem of poor code quality than is carving off large sections of time for exclusively improvement-oriented work.

Acceptance Test Driven Development

The book’s worth it just for this material alone.  Larman and Vodde present a wonderful analysis of ATDD and characterize its place in the context of Scrum, including tying it into iterative requirements refinement, the Definition of Done and the Sprint Review.  I’d seen teams go in this direction driven largely by the desire to better engage testers in their teams throughout the Sprint and avoid “scrummerfall.” The authors take this further to show how ATDD, in which acceptance tests are defined and automated in advance of the development of the code that will pass those tests, does for teams in an iteration what Test Driven Development (TDD) does for individual developers in ten minutes.

They stress the need for the test automation to be a distributed responsibility shared by the whole team rather than one assigned to a specialist team.  I couldn’t agree more.  I’ve seen many attempts to establish test automation through the creation of a group apart from development and I can’t say that I’d call any of them particularly successful – at least not in the broad and encompassing sense that Larman and Vodde are envisioning in this book.

Continuous Integration at Scale

I was particularly glad to see treatment of Continuous Integration (CI) at scale. I’ve seen groups start with vanilla CI as it is described in the Extreme Programming literature and do well with it initially but then fail as their groups grew larger.

Essentially, Larman and Vodde propose a nested set of continuous integration builds, each larger one subsuming sets of smaller ones within it and each build defined for a particular concern.  Examples of these concerns include the verification of specific components and subsystems but also particular kinds of testing – including things like performance and stability testing.  One key aspect of this approach is that, at each level, you’re trading off the immediacy of feedback to the developer against the likelihood of the developer’s submission affecting quality at that level and the expense of the tests.

The Elusive Definition of Done

As noted earlier, Larman and Vodde,  suggest defining “done” as that subset of the work needed to release the product the team is currently capable of performing each and every Sprint – then use your retrospectives to challenge the team to bring more and more of the “undone” work into each Sprint.  The authors point out that there are really only two ways to extend the Definition of Done:  increase the cross-functionality of the team or increase the degree of automation the team uses.

In the meantime, however, while the Definition of Done leaves some work undone, Larman and Vodde suggest meeting that problem honestly, by doing things like defining an undone work team and pushing undone work onto the Product Backlog.  By being explicit about where the team currently stands against the goal of delivering potentially shippable product increments, this undone work can be better managed and the team’s insights can be directed to the problem of expanding that definition of done to get them closer to the goal.

For the Bookshelf of any Agile Team Member

The book isn’t without its faults.  It’s certainly long enough and some sections can be tedious (there’s a fifteen or so page section where different scenarios for story splitting are laboriously addressed – I get that this common complaint about stories needs to be put to rest but this feels like killing me with kindness :)).  The book’s organization lends itself more to use as a reference than as something you’d read cover to cover.   There are many repetitive sections that allow each chapter to stand on its own but are a bit hard to get through when you read it straight through.  But these are really just quibbles.  In all, “Practices for Scaling Lean  & Agile Development” is, just as its companion volume before it was, a tremendous book that belongs on the bookshelf of any agile coach, manager or team member.

I’d like to thank Anne Greenhaw and Selaine Henriksen for their help in developing this post.  Thanks also to Ryan Martens for inviting me to post here.

About the Author: Ed Willis has been a ScrumMaster, Product Owner, Scrum coach and trainer.  He is currently a developer in the telecommunications industry.

Managing the right goals in software development can have a major impact on your team’s success. You get what you measure. In this regard, the goals you choose can be a defining factor in whether your organization’s Agile adoption thrives or dives. So, what are your choices?

A story about goal-setting

Imagine you are a runner. You have a running coach defining targets for you. Your coach chooses miles and seconds in order to measure you. These metrics turn out to be useful; they are universally understood by you, your coach and your competitors. That’s a good thing. A target goal exchange around these metrics might look like the following:

Coach: “I need you to run a 2-minute mile.”

You: “Uh, I’m not sure I can do that. In fact I don’t think it has ever been done before by any land animal other than a wildebeest, a pronghorn antelope or a cheetah. And, I’m not a wildebeest.”

Coach: “You’re not listening. I already told Sports Illustrated you will. Don’t make me look bad.”

You: “Seriously I’m pretty sure that’s just not humanly possible. I’m no Usain Bolt and I’m not sure even he could sustain a 2-minute mile.”

Coach: “Hey, I set the goals. Run that mile in 2 minutes. Just hunker down and work harder. 2-minutes is the target goal. If you don’t meet it, I’m kicking you off the team.”

You: (muttering under your breath to self, “I can’t do it, you aren’t listening to me, and you are a moron. Why do I even bother to open my mouth?”)

Why goals matter

Get the idea? I’ve been reading John Seddon’s Freedom from Command and Control: Rethinking Management for Lean Service. My colleague Alan Atlas recommended the book. Then my manager Ryan Martens became very interested which led to a book appearing on my desk. Soon after, Karl Scotland mentioned it in an informal conversation about Lean and Kanban. Time to pop it to the top of my reading stack!

Seddon sets a context about Purpose driving Measures that then drive Methods. The combination of purpose and measures start to look like goals. And Seddon breaks goals into two types: Target Goals and Capability Goals. While John’s book is not specifically about software, I could see how we use and abuse goals in our software world. So I decided to delve deeper.

Target goals are arbitrary and doom-laden

Target goals are set to measure hopeful results, often an arbitrary number. Think about our runner story and the 2-minute mile. That target goal was  set by someone else. A target goal typically does not account for your strengths, capabilities , availability and skills. Target goals don’t serve you. They don’t serve organizations either. And yet we cling to them. Consider if, as a developer, I hear the following from my manager:

“Jean! You missed the release deadline we gave you. We told marketing and sales who then told our customers that dev would deliver this specific functionality on this specific date. You’re ruining everything. We’re doomed I tell you, doomed! And all because YOU didn’t hit YOUR deadline.”

Uh, it wasn’t my deadline. And yet, missing that goal is now rippling negatively through the organization. This is a lose-lose situation all due to expectations hinged to an arbitrary target goal.

So why do target goals exist?

In many hierarchical, command-and-control software organizations, target goals are how deadlines are set. As Seddon points out, use of ineffective target goals usually arises due to an impossible purpose coupled with non-value add methods. The measures (or goals) are meant to drive teams to the impossible purpose. A command-and-control organization believes that goals must be set top down. The negative impact of these goals is inextricably woven into unrealistic expectations, death marches, and non-value add methods for controlling the work toward the goals.

Enter the capability goal

Let’s replay the runner scenario again.

Coach: “There is a race coming up, a half marathon. That means we need to start training now. What is your time currently?”

You: “I haven’t been really running lately. Rather than guess my times, I’ll start running now. Do you want to know in terms of how long it takes me to run a mile, or how far I can run in a specific timebox?”

Coach: “Let’s start with how you finish a mile. What is your typical way of getting going?”

You: “First, I do a walk/run combination. Then I can give you my actual mile capability in a week.”

Coach: “Okay, let’s set up a plan and together set some sort of goals based on your capabilities. That will help me guide you and figure out when we are faltering. I know you can do this. I’m going to work with you starting right now.”

You: (muttering under your breath, “This is amazing; yea me! I am going to finally put my passion for running to work, keep improving, and run my first half marathon. I love my coach!)

Capability goals are based on real data

Together, the runner and the coach watch the capabilities of the runner to define goals. The purpose of the goal is clear. Measures are set up that usefully help determine the runner’s progress. The method (or running regimen) for attaining the goal is then applied. Simple.

We use this same approach in Agile teams. A team lead or project manager does not set an arbitrary target goal. Rather, the Product Owner sets an overall purpose/vision. The Agile lead, often referred to as the ScrumMaster, works with the team to assess what goals can be reached based on a team’s capabilities and availability. From iteration to iteration, the team provides feedback on what it is able to complete based on its capabilities. The Product Owner absorbs this information and sets expectations outside the team. The process of continuous improvement continues. The team declares, “Based on what we tend to complete, and given the purpose from the Product Owner, we are going to improve our methods in the following ways.”

When good capability goals go bad

There’s a Gary Larsen “Far Side” cartoon of an open refrigerator with a bowl of glop inside holding a gun. The caption? “When good potato salad goes bad”.

Unfortunately, I’ve seen good capability goals go bad in a number of organizations. Listen to this conversation:

Program Director: “I see that our Agile teams are all now using these things called story points to track their work.”

Me: “Yes, it’s working well. Each team has a good sense of what it is capable of delivering iteration over iteration. That helps me figure out how to help remove impediments, where bottlenecks are, and where any team may need us to better resource them. It’s great!”

Program Director: “Hmmmm.”

Me: (muttering under my breath: “That ‘Hmmmm’ just doesn’t sound good. I think I’m about to get sick.”)

Program Director: “So, what’s the highest story point commitment a team has made?”

Me: “27.”

Program Director: “Great!  I’m setting 27 points as the target goal for all teams to commit to for each iteration. Every point will represent 3 hours of work. This precision will help me calculate exactly what we’ll deliver when. Plus, I’ll know which teams are really working and which ones are slackers.”

Me: “Hmmmm.”

Program Manager: “So, let’s get out there and commit! This Agile stuff is great!”

Me: (muttering under my breath, “We’re doomed, I tell you doomed!”)

What just happened must not be allowed

As you roll out your organizational Agile adoption, you’ll encounter many challenges. Pay attention to the usefulness of capability goals. Insist on using them.  Non-Agile organizations won’t like this. At times, you may feel like Sisyphus pushing a rock up hill for all eternity. Be strong. Don’t allow Agile measurement abuse. Create team and organizational trust.  Remove target goals and embrace the incredible value of capability goals. And, don’t allow any backslide. Don’t let your capability goals devolve slowly into target goals in sheep’s clothing.

You: (muttering under your breath, “You know? This MAY just work. Capability goals versus target goals. For the sake of my team, our organization, and the value we deliver to our customers, I’m going to do it!”)

Yesterday, the Agile Journal posted their May edition of the online publication with the above focus.  To support their release, they asked us to share our opinion on the topic.  Here is our response:

Ryan Martens: Age of Reconsideration, Reform and Regeneration

The last decade marshaled in a new empirical way of working with increasingly complex, interconnected and highly-critical software-based systems – Agile.  We are entering a period of reconsideration, reform, and regeneration in software, systems engineering and project management.  Agile is working, Lean Software and System is working, and the combination is starting to prove very powerful with regard to throughput and workers.  The benefits of autonomous work, engagement and mastery are driving systemic improvements in our way of working and growing to meet complex challenges of our world.

These results illuminate a future vision that has the potential to expand our current notions of Lean and Agile from software teams and into real organizational agility.  As a result, there is a chance to unite and unify many communities under the guiding ideas of flow, pull, and value.  All of these communities are being drawn in and starting to play well.  These are beautiful days with all the implications to CMM/SEI, Agile, Scrum, Lean/LEI, and PMI/PMBok communities yet to be determined.

In the first half of this decade, look for collaborating across boundaries, seeing larger systems and groups working hard to create their future realities.  Following that period, look for messy consolidation as the winners of this new platform emerge for a new golden age of networked, software product and system development. Together we’ll be focused on the problem domain of global scale difficulties in governance, cyber-warfare, energy, water, communication, commerce, medicine, climate, transportation and nano-technology.

Ryan Martens is an amateur triathlete,  founding board member of the Entrepreneurs Foundation of Colorado, and CTO at Rally Software Development.

Jean Tabaka: Let the System Talk

Thinking about our path with Lean, I’m compelled to draw upon research I’ve been doing in Systems Thinking and, more recently, what I’ve been learning in Systems Engineering.

In Systems Thinking, we recognize a world of system archetypes based on the dance of balancing loops, reinforcing loops and the outside agents that may cause them to transition. Lean, as a system of thinking, has certainly responded to systems that rely too much on take-make-waste. A set of negative reinforcing loops: the more you waste the less you have to take and make. Outside agents, the scarcity versus abundance of materials, has led us to Lean. Lean principles and practices create a positive system wherein the more we reduce waste the more value we get which in turn reinforces more waste reduction. It is a reinforcing loop propelled by continuous improvement.

Recently, I attended the Lean Software and Systems Consortium’s 2010 conference in Atlanta. What a revelation. From James Sutton’s talk on Lean Systems Engineering, I added new vocabulary that I think will become critical to Lean’s future.

Will Lean be our best source of practices and principles in the future? That depends on what will be guiding our systems:

• Scarcity
• Abundance
• Desperation
• Conformity

Once we have clarity about what guides our system, we can understand more about the system in which we are operating:

• Simple
• Complicated
• Complex
• Chaotic

Lean has steadfastly addressed pressures of scarcity and hence a system of complexity. That brings me to Dave Snowden’s work captured in Cynefin, a Welsh word he has used to describe a framework of problems, situations and behaviors in these four systems. For our world of complex systems, Lean provides the perfect high-level thinking for what we must embrace: emergent practices informed by, as Snowden puts it, “sense-making”

As we move into the next 10 years of Lean, I fervently believe that our sense-making must inform us about what supports emergence that responds to complexity. The practices will follow. For now, let us concentrate on the systems in which we operate, what outside agents or pressures are guiding our systems, and how we can best continue to formulate and hold dear the practices that will naturally emerge.

Jean Tabaka is a wine enthusiast, author and Agile Fellow at Rally Software Development.

Last week, 10 of us from Rally product development, sales and coaching attended the Lean Software and Systems Consortium’s 2010 conference in Atlanta.  For me, the learning highlights of the conference were in the Systems Engineering track  led by James Sutton. To kick-off his talk on Lean Systems Engineering, James used a number of compelling stories around different systems and what guide them.  As part of his introduction to systems, he played this hilarious Dave Snowden video about how you plan or act based on one of 3 systems in which you might find yourself -- Chaotic, Ordered and Complex.

David Snowden -- Cognitive-Edge -- Ways to Plan a Party

James, a Systems Engineer, Principal with Lockheed Martin, is an invested member of the Lean SSC board as well as a technical advisory member.  He is the deepest expert that I have ever met in the field of systems engineering. And he has a wonderful gift for bringing multiple resources to bear in helping people understand and care about systems. For a Civil Engineer and Computer Science major like myself his talk and track were the definition of a prop-spinning geek nirvana.

Systems are guided by the pressures that form them

The point of view that James imparted on us was to understand that there are fundamental systems in which we operate. They are not value-based; one is not better than the other. They just help us set context and inform us about the world around us. Why should we care about this? How you approach your product and project development depends on which system  you find yourself in. And, as it turns out, the system you find yourself in is largely guided by one of four compelling pressures. That is, you will recognize the system in which you operate based on what is driving you to act.

There are four fundamental pressures that guide us: abundance, scarcity, desperation, or conformity. And each leads to a different system context. To illustrate this, James led us through the story of four different nations following the second World War.  Each nation, responding to different drivers, led to advances in different types of systems management approaches in use today.

• United States -- Abundance -- Systems Engineering

• Japan -- Scarcity -- Lean

• England -- Desperation -- Chaos Theory

• USSR -- Creativity and Conformity -- Patterns of Inventiveness
  

The Four Systems

Given this sense of what guides particular systems, James explained that we live in a world of four fundamental systems: Simple, Complicated, Complex, Chaos.

Cynefin

Once you recognize what system you are in, you discover what principles and practices will best serve you in that system. But systems tend to not be static. So, James presented what agent or pressure might cause you to move to a different system and therefore what tools and practices would guide your thinking and actions for transition.

For instance,  if you are in a Simple system and are moving into a Complicated system, Lean Manufacturing and Analytical Systems Engineering are your best tools and guides. If, however, you are in a Complex system verging on Chaos, you’ll work best relying on the perspective originated by Dave Snowden: Cynefin, the Welsh word for “the place where you hold multiple things.” Cynefin serves Complex systems well as it emphasizes emergent behaviors and, what Snowden refers to as “sense-making.”

The history and vision from this talk became almost a grand unifying theory for me. It all made great sense. If you are a  systems engineering fan, do not miss the recorded version of this talk when it becomes available.

Thanks Lean SSC

While 6 speakers and several attendees from Europe were prevented from attending the conference due to the Icelandic volcanic ash cloud, the Lean SSC rolled with the punches and pulled off an excellent event. The folks able to attend and the over 40 sessions offered created an electric buzz both in the air and on Twitter. Given the caliber of sessions, hallway discussions, and Open Space,  I am sure there will be many posts that emanate from attendees. And no doubt new ideas will be growing that were nurtured by the conference. Kudo’s to the Lean SSC board for creating a space for this excitement and emergence.

About the Authors:

Today we announced Rally’s acquisition of AgileZen, a visual project collaboration tool that manages work using the Lean concept of Kanban. AgileZen is a simple, elegant project collaboration tool that supports software development by providing a Web-based Kanban board.

Our definition of Kanban

If you aren’t yet familiar with Kanban, there are lots of great resources out there. The simplest description we could come up with for Kanban and Scrum is in our press release, but we welcome your thoughts and additions:

Kanban literally means “sign board,” and in Lean it is the signaling tool for visualizing and tracking work as it flows through various stages of a process. A Kanban board does this by exposing bottlenecks, queues and waste in a process so that teams can deliver high quality, high value work. Both Scrum and Kanban methods focus on early value delivery, and both provide transparency into the work in progress.  But Kanban can operate with a different planning and delivery cadence than Scrum and emphasizes different metrics.

What does this mean for Rally and the industry?

We believe that Kanban is a simple but natural extension of Agile software development. It will invite more teams into Agile and provide more runway for mature teams. But most importantly, it will help us extend Agile beyond development teams to create an Agile business. The AgileZen team has been effective in all of these areas. We are in heavy learning mode and, at least in our view, the entire industry is still figuring out how Scrum and Kanban work together and which methodology is better fit for various projects.

Welcome Nate and Niki Kohari!

Nate and Niki Kohari, co-founders of AgileZen, have built the best Web-based Kanban board out there, and we have the utmost respect for their product, company and brand. We are incredibly excited for them to join our North Carolina office.

What does this mean for AgileZen?

First, you should read Niki’s blog post. Not much has changed for AgileZen users. Together, we’ll continue to support the AgileZen solution as a low-cost Kanban-focused project collaboration tool, and users can access support as they always have.  If you aren’t familiar with the AgileZen product yet,  check out their free product.

Join us at the Lean SSC Conference in Atlanta next week!

The Rally and AgileZen teams will present our products and coaching services at the Lean Software & Systems Conference 2010 in Atlanta April 21-23. Ryan is also speaking on Plan-Do-Check-Act. We look forward to seeing you there!

About the Authors: Ryan Martens is a goat cheese maker,  founding board member of the Entrepreneurs Foundation of Colorado, and Founder and CTO at Rally Software Development. Jean Tabaka is a wine enthusiast, author and Agile Fellow at Rally Software Development. Subscribe today to get free updates by email or RSS.

On April 21 in Atlanta, the Lean Software and Systems Consortium will come together for its second US conference.  Last year’s event in Miami was “amazing” according to Jean.  So this year, Rally is exhibiting, I am speaking while Jean and Aaron are running the open space on Friday.  The price per attendee goes up by $250 on March 31st, so if you do intend to go, REGISTER now.

At our booth, Rally will be demonstrating its product support for highly-visible Kanban, WIP/Cumulative Flow reports, and cycle-time  metrics. Join Alan Atlas, Jean Tabaka, Aaron Sanders and Craig Langenfeld in our booth.

I will be presenting an experience report titled: PDCA: Beyond Simple Inspect and Adapt. On spring break this week, I’ve been thinking more about the details of my talk. Here is my abstract and outline for those of you who might consider attending:

Lean and Kanban focus on practices of continuous flow of product delivery. Plan-Do-Check-Act (PDCA) is a Lean discipline that moves beyond inspect and adapt of Agile team-level processes.  At a corporate level, PDCA provides guidance for strategy as well as problem-solving work. In 2009, I led Rally’s move to PDCA for the company’s strategy process at both the annual and quarterly levels.  My primary guide was Pascal Dennis’ “Getting the Right Things Done”. In this experience report, I share Rally’s PDCA first year of adoption: where we started, how this impacted our corporate behaviors, and where we are now. I want to share Rally’s story to compel participants to embrace PDCA and get good at it. I ask each participant to come with its organization’s #1 goal and success criteria. I will close with a planning A3 exercise from Pascal’s book .

Outline

1. What brought me here—background on why I am passionate about sharing my organization’s overall Lean story including the addition of PDCA, A3’s and concurrent set-based development.  This talk focuses on PDCA as the critical step in increasing structure and discipline in strategy execution.
2. Point of View – Use PDCA to move your planning horizon out and as the principle governing mechanism for organizations in continuous flow.
3. Benefits — Mature your strategic planning and execution environment to handle the complexity of increasing speed, agility and scale and to gain alignment, pull and innovation.
4. Where we were and what was not working
   1. The context at Rally was based on a couple of key concepts:
      1. Core Values, Core Purpose, Sandbox and BHAG from Jim Collins
      2. 3 to 5 Quarterly Rocks, success criteria and Scoreboards from Gazelles
      3. Rock team structures  – cross departmental and story based
      4. Facilitated, highly collaborative cross-departmental meeting of 30+ managers and above
   2. What we noticed
      1. Rock work as a second or third job
      2. Wisdom of the Crowds to help fix over-reaction and group think
      3. Too much on the fly, not enough backlog grooming
      4. Highly critical, non-cross departmental initiatives were de-prioritized
      5. ORID process added to keep from jumping too solutions, but the data was not visible enough
5. What we decided to do about this:
   1. Explanation of PDCA — A brief overview of PDCA in general and then specifically what I used as guidance from the Pascal Dennis book, “Getting the Right Things Done: A Leader’s Guide to Planning and Execution”.
   2. Our initial experiments with A3 process the year prior — Working with our Ops team and product marketing teams on problem solving using real data
   3. First quarter — How we kick-started Rally’s company-wide adoption of PDCA . I describe our “Mountain Team” and their transitional role.
      1. Defining Rally’s True North
      2. Creating our second level tree with current and needed metrics
      3. Socializing these throughout the company seeking feedback in anticipation of our annual and quarterly planning
      4. Started new experiments based on quarterly planning decisions
   4. Next Quarter – Review new experiments, discussed learning and drive A3’s into the planning process
   5. Mid-course adjustment by Mountain team, in middle of the quarter – What we discovered working and not working
      1. The rocks were all dependent on each other.
      2. Had to run Rock of Rock team meetings to steer to a final solution
      3. Coordinated release planning would have
   6. Final quarter – We worked to expand the plan. We took the Mountain team’s True North and feedback to drive our PDCA for Rally’s Annual Corporate planning by:
      1. 1. Taking company-wide feedback into our Annual planning to collaboratively drive cross-department A3 creation around each branch of the tree
         2. Mountain Team retrospective over the course of year 1 that helped create a planning rock team.  The Mountain team’s role as a transition team ended.
   7. Year 2 – Doubling down our efforts to go from amateurs to intermediates —Changing our process to institutionalize A3’s and PDCA as our strategy execution approach:
      1. Quarterly rocks moved to a world of pre-defined from developed on the fly
      2. Quarterly planning moved from ad-hoc based on yesterday’s weather to more programmed based on True North and meeting the target metrics
      3. Strategic planning worked to validate annual True North in the context of long-term planning, shared vision development, cross-boundary collaboration and larger systems
6. What we learned and what you should do about it
   1. The cycle of adoption is a year, quarterly cycles work to improve the process, but it is hard to make leaps on a quarterly basis.
      1. Year 0 – Introduce Lean thinking (A3 in our case)
         
      2. Year 1 – Introduce PDCA (Novice)
      3. Year 2 – Invest or abandon (Your choice)
         1. A3 is now the language for problem solving
         2. Making sure we are solving the right problem (aka slowing down to speed up)
   2. Do not have a overall guidance team steering the continued PDCA process – it is owned by the “team”
   3. Putting pressure on the organization to get more clear about our economic models to mature from “Theory-based decision making toward the right solution,” Now “Data-driven decision making toward the right problem”
   4. Where to start – The Strategy A3 an exercise
   5. What is next? – We call it the Innovative or Lean Organization. Seeing large systems, collaborating across boundaries and creating your reality.
      
7. Point of View – Use PDCA to move your planning horizon out and as the principle governing mechanism for organizations in continuous flow.
8. Call to Action – Introduce the language of A3’s through problem solving or Strategy A3’s
9. Benefits — Help build a company of problem solvers to focus your efforts on the critical few things.
10. Where I hope you go with this: Great companies build great software, great experiences and work on creating win/win scenarios.
    

I’ve noticed a piece of advice repeated in many Agile blogs, articles, and books.

Seeing it makes me roll my eyes until it hurts. (Why I would hurt myself on purpose will be the subject of another post, on a blog reserved for psychotherapists.)

Even my very most favorite Agile book, “Scaling Lean & Agile Development” by Craig Larman and Bas Vodde, has a section in there with this advice.

I saw it in Jim Highsmith’s new book, too, although by the time he’s done discussing it he does make a couple of good points. It’s an old piece of advice that pre-dates Agile.

What is this old chestnut? Here it is:

Hire the best.

I mean, come on. Is this supposed to be a big lightbulb moment? Where do they find stuff like this, in the “Journal of the Totally Obvious?” Am I supposed to leap out of my chair, smack my forehead and exclaim “Eureka! All I have to do is hire the best! Why didn’t I think of that?”

Is this really good advice? Is it actually possible, or necessary in an Agile world? Is this sensible, if trite, piece of advice useful at all? Talk amongst yourselves while I blather on for a bit.

The problem with “Hire the Best” as an operational principle is that:

a) it immediately excludes most of us, and
b) it’s extremely difficult to do.

What’s the best? The top 5%? 10%? Certainly no more than 20%.

So what about the rest of us? What are we supposed to do? Are things hopeless for us? Should 80% of companies worldwide just give up and shut down because the top 20% of people are taken? What about big companies and the Law of Large Numbers? Can you really hire only the best when you’re hiring 10,000 or 20,000 people?

Something that makes much more sense to me, and which has much more power, is this idea:

Hire well, and develop people.

Check it out! Everybody can do this. “Develop People” is one of the two pillars of Lean, while “Hire the Best” is not. So far, those Lean folks have been right about pretty much everything, so why not this, too? Why would I need to develop people if I only hired the best? Why not save the money so I can pour it into my “Hire the Best” employment initiative?

The Agile Principles say something like […find motivated people and trust them…], and I believe in Agile. So I cannot find in the bedrock of either of my professional beacons, Lean and Agile, any indication that I should “Hire the Best”.

My common sense and experience tell me that it is incredibly hard to actually hire the best, and I like that it might not be absolutely necessary for success. How cool would it be to hire the ‘pretty good’ and then kick the butt of some company that thought it was hiring the best? Is that possible? Yes.

“Hire the Best” is really hard to do.

I’ve worked as a full time hiring manager at more than a dozen companies, all of which thought they hired the best and only one of which actually did. That company really worked hard at hiring the best. At that company, one rule of the hiring thumb was that you only hired people onto your team who would immediately place in the upper 50th percentile.

In other words, when you were on an interviewing team in that company, you were expected to vote to hire somebody who was better than half the people on your team. You think that’s easy? You think that isn’t scary? Try it sometime.

What I’m really really really interested in is something that can take my average team (and let’s face it, Ms. Wishful Hiring Manager, it is overwhelmingly likely that our team is average, for some value of average) and make it better or improve its ability to deliver value to my customers. That’s worth some effort to achieve because it is worth money to my business. If I believe in “The Art of the Possible” then I like this better, because it’s a lot more possible than simply hiring the best.

Anybody can embark on a long, expensive and likely unachievable quest to only hire the best, but if Agile were really valuable,  it would help me to take my team of competent professionals and make them significantly more effective than they were. It might even make them more effective than a gaggle of “the best” somewhere else.

The Agile Principles talk about motivated people, but they don’t actually mention “the best”.  I view this as a good thing because I strongly believe that the best teams are not built from a homogeneous mix of the smartest, fastest, bestest people.

Teams work best when they are diverse and when the power of the team can be unleashed by empowerment and self-organization. I also know from bitter experience how hard, and frankly scary, it can be to really hire the best. (Sorry if I harp on that, but I have scars…)

What I want is what I think both Lean and Agile offer to me as a businessperson. They offer me a way to take solid professionals and then ignite their passion and professionalism within a framework of continuous learning in a way that allows them each to contribute to the fullest extent possible.

That’s something that make Lean and Agile worthwhile to me, and not some lazy idea about hiring the very best (somehow) and then automatically winning.

About the Author: Alan Atlas is a Soul Musician, Certified Scrum Trainer, and Agile Coach at Rally Software Development. Subscribe today to get free updates by email or RSS.

Looking for the perfect trend article to pass on to your executives about Agile development? Check out Dan Woods‘ well-researched, executive-level article “Why Lean and Agile Go Together” on Forbes.com Jargon Spy.

Dan describes how the Agile impact in software is similar to the Lean impact in manufacturing of the 90’s.  I initially struggled with the analogy until I realized he is not saying that software development is like manufacturing. He is saying that the concepts and techniques applied in Lean manufacturing are coming to large-scale software development.

I completely agree and believe the model of how we build software is changing just like Dan describes.  Using the model from the Fifth Discipline Fieldbook, the change looks like this:

Lean Software Concept

That was a great decade – I see much of the last decade of improvement coming from a real effort to take responsibility for how we perform our craft.  The ground up work in Agile, Scrum, XP and Lean has increased velocity, increased quality and increased the quality of life for many in our industry.  I am personally in a much better working environment; how about you?

So what’s ahead for 2010?- With Agile software development hitting the mainstream, the guiding ideas on how to design and deploy software is changing for most software driven organizations.  As more teams and organization make the transition to the Agile/Lean triangle, I see us increasing our investments in skills training, teaching, experimenting and innovating with an increasing number of new tools, techniques and methods.  As a result,  I see this decade moving us past the Lean models of Toyota to something uniquely our own.  I am really excited about the improvement this decade could bring to our industry and all the industries that we impact with our software.

About the Author: Ryan Martens is a goat cheese maker,  founding board member of the Entrepreneurs Foundation of Colorado, and Founder and CTO at Rally Software Development. Subscribe today to get free updates by email or RSS.

For 2010, lets find ways to focus on teaching our craft and growing the world of skilled software development professionals instead of trying to figure out who is “right.”

I believe much of the “Escalation” that Jean is seeing was correctly titled by Regina Mullen as a battle to be “right.” (see and read Escalation is Killing Agile – Can We Please Stop It? and Escalation is Killing our Healthy Conflict in Agile). That behavior focuses on carving up the pie instead of growing the pie.   There has been so much added to the field of software development methods, tools and techniques from the guiding ideas of Agile.  Now is not the time to stop and eat.

For me, 2010 is about continuing to grow the Agile software development pie’s reach and innovations.

I believe one of the key fixes to the problem of escalation can be found through increased professionalism and certification in Agile. By raising the bar through “difficult and skills-based certification,” as Brian Marick and the board at the Agile Alliance described, we can advance the Agile discourse through :

• a defined a bar that is deep in skill, knowledge and practice
• a significant enough bar to engage College and University study and examination
• research and curriculum that explore the tough questions in a scientific method
• development of more flexible or “T” shaped individuals that can see and work beyond silo roles.

With this back-drop, I am motivated by the notion of creating a  A Community of Thinkers,:

I am a member of a community of thinkers and I believe that communities exist as homes for professionals to learn, teach, and reflect on their work.

A Community of Thinkers creates more leadership in our profession. I see the expanding certification efforts in 2010 as great steps in these directions:

• Lean Software and System Professional – new certification emerging
• Agile Developer Certification – that is also emerging
• Certified Scrum Practitioner – especially as it intertwines with the PMI’s Agile Community of Practice

I encourage everyone in our community to figure out how to put energy toward one or more of these efforts.  The benefit of actively learning, teaching and reflecting on our work should lead us all to expanding civil dialogue that works to understand all points of view and keep expanding our thinking.  Thus broader education and difficult certification helps create a “Community of Thinkers.”  And, a Community of Thinkers will create a virtuous cycle of win/win and thus a larger pie for all.

That is my hope for 2010 in our profession.

About the Author: Ryan Martens is a happy father,  founding board member of the Entrepreneurs Foundation of Colorado, and Founder and CTO at Rally Software Development. Subscribe today to get free updates by email or RSS.

Recently, I wrote the post “Escalation is Killing Agile – Can We Please Stop It?”  My passion around escalation brought 40+ in-depth comments.  With my travels and lack of internet access, I had been unable to sit down, sift through, and absorb all the various perspectives.  Until now.

I’m offering this follow-up post as a means to provide an overall response to all these great comments. I want to add some further background on the “escalation” topic and some more of my personal conviction around it. Specifically, I’d like to provide some insights into delayed feedback, the need for conflict, and how to “show up”, all without escalation.

In one part of the comment stream, I heard and felt the call for an effort to get to the root cause of such deep-rooted assumptions.

According to the Systems Thinking Toolbox from Pegasus Communications, to break an Escalation structure, ask the following questions:

1. What is the relative measure that pits one party against the other and can you change it?
2. What are the significant delays in the system that may distort the true nature of the threat?
3. What are the deep-rooted assumptions that lie beneath the actions taken in response to the threat?

So, in our system of sharing information in the Agile community, we have to ask, “Are we setting up a dynamic that pits us against one another?” If this is true, then we have to ask, “How can this be addressed and still ensure that we share insights?”

Guided by Systems Thinking, that means we need to check in with: what is distorting our communications and what might our deep-rooted assumptions be that would have us act as we do?

Here is an example:

I created a delay in feedback by my not responding to posted comments. I believe that created assumptions around what I may or may not have intended in the post.  I think some individuals thought I had written the post pointed specifically at them. Faster feedback would have helped quell that assumption.

I was writing about, and continue to write about, the Systems Thinking Escalation archetype and how I see it in our community. I was and am looking at a dynamic not at an individual. Escalation is NEVER about an individual; it is about the system in which blame is occurring and allowed to continue. I am fearful that blame and the win/lose game are in our system and that is what I do not like and I want to address.

Some of the comments to my post seemed to indicate that I was anti-conflict. Far from it! In studying the inner workings of high-performing teams, I have often referred to Sam Kaner’s model for participatory decision-making. Conflict is a must.

In this model, Kaner insists that, to get to high performance we must bring forth conflict to discover the best informed decisions. Divergent ideas must be invited. Divergent voices must be heard. Divergence must be allowed to just be. That is, don’t just jump to conclusions. With enough time and patience around divergence, we can then move toward informed convergence.

Conflict in this context is dialogue. It seeks insights. It invites greater and greater participation. I also want to emphasize that in this context of dialogue and non-escalation, our purpose is to engage in forward thinking. We let go and we look forward. And as we look forward, we let go.

So, as a member of the Agile community, my interest in expending energy in discussions is to seek insights, encourage divergence, and discover convergence as it emerges. All of these practices help and encourage me to create more and more forward thinking. If this is not occurring in our community, then we are not getting enough for the energies we expend.  If in our community we really “must win”  in order to “be heard”, we are stuck in an “Escalation” archetype. And, that means we are all trapped on an up escalator to nowhere.

What could any individual do to break an escalation pattern in a system? Create energy around your insights and share them without a need to apply win/lose stakes. Stop expending energy to refute others.

Here is a simple formula for bringing your viewpoint to bear without escalation:

1. Show up. (Be willing to be engaged.)
2. Find out what has meaning to you. (Be willing to be honest about your perspective.)
3. Tell your truth. (Be articulate about your insight without attacking or assigning blame.)
4. Let go of what happens. (Be courageous enough to allow others to agree or disagree.)

I believe this formula provides guidance on how to remain forward thinking, remain open-minded, and remain engaged.

I have some more mental models I want to offer here. But they will wait for another post.

Thank you in advance for your considerations, insights, and comments.

About the Author: Jean Tabaka is a wine enthusiast, author and Agile Fellow at Rally Software Development. Subscribe today to get free updates by email or RSS.