Agile Development Blog

Question: How did you get started from waterfall?

Answer 1 -  Started with whiteboards and sticky notes.

Answer 2 – Started with in-house tool, then open source and finally Rally.

Answer 3 – Started with Rally training for the first year with a Rally pilot in parallel to show management and staff the whole solution.

Answer 4 – Put good consultants alongside the team from day 1 to apprenticeship with the best – we went Rally training and consulting.

Question: We have lots of concurrent features in parallel. Does anyone do concurrent feature branches?

Answer 1 – Peggy – At Avaya we do this, but first go back to the product owner to prioritize better.  We use private views in ClearCase to manage features branches with continuous integration in the views.

Answer 2 – Israel – Never allow the backlog to be larger than twice the capacity of a release.

Question: Please talk about the adoption challenges of adopting with your external customers.

Answer 1 – Israel – It took us 18 months to change the perception of my group with sales.  It was critical to get field engineers to come to release planning and every iteration demo. They get exposed to the software engineering approach and real status and built the best evangelists and advocates in the process.

Answer 2 – Richard – Rally has developed a social networking site with what most people see as shocking openness.  As a result, we have had tremendous success in engaging customers and focusing priorities.

Question: What is the cheapest and best way to get started as an individual?

Answer 1 – Take trips to visit other local companies – everyone says yes.

Answer 2 – “XP in Trenches” – was recent great book find- read a ton of books but most only focused on single team.

Answer 3 – Richard – Come be a fly on the wall at a Rally release planning.

Question: Big Six Sigma background – What tools do you use QFD to find what customer really wants?

Answer 1 – Failed with all upfront tools, only thing that works is communication in every iteration.  Too many edge cases in our industrial software.  Given up trying to spec this in advance.  Invest more in bringing the customer in.

Answer 2 – Israel – Try to reach the point where the developer can code faster than marketers can release.  As soon as that happens, you can decouple and get more feedback during the cycle.  You are also able to bring more of the business into learning after you decouple.

Question: How do you solve the coordination of many teams in a large adoption? Tell me more about Meta Scrum.

Answer 1 – We do a meta scrum every week with the owners of seven product lines.  In parallel the product owners and development leads do the same.

Answer 2 – Israel – Scrum of Scrums everyday with all the ScrumMasters following a 10 to 15 minute approach.

I’m live here at the Denver Agile Success Tour on yet another warm, blue sky day downtown by the Convention Center.  I will blog what I pick up as the key takeaways from our speakers and after the event we’ll get them to chime in with their own thoughts.

Israel Gat – Cutter Consortium member on Agile, former VP from BMC, IBM, Microsoft and EMC

A Modus of our Time – The industry has “One foot in cold water and one foot in hot water”

Tons of enthusiasm about Agile given years of outsourcing, while also having lots of hard data showing how time-to-market, productivity and quality are improved with Agile.

But, still falling short by not pulling in the customer.  We assume we know the customer problem and work hard to develop the solution.  The speed and reach of software is immense.

Ray Bagley – Director of Development of 3D Modeling solutions at Spatial

Spatial – 20 year old products with 4 million lines of code. Developers spread across 3 offices (Colorado, UK, Pune). Top-down adoption started with a new VP two years ago.

“Product Owner is not Product Manager” – Read all the books on Agile. Early on divided up the product managers to support multiple scrum teams, but tons of pathology happened. The Product Managers stopped looking outward!  Then, a collision of two product managers on one product.  Finally, the senior and highly skilled development team felt like they lost all their decision making and influence.

Iteration 2 – Created Meta Scrums that met every two development iterations and life got much better. Added more product owners. Life is much better thanks to Rally and Enthiosys content -  I wish we had found that material two years ago!

“There is a difference between a real customer and a virtual customer on two different products” – “Real customers engage on day 1, not with a beta!”  The case of these two products was night and day – night (solved the wrong problem after 8 months) and day (driving revenue from the first release).

“Humble lessons learn the hard way!”

Lloyd Star – VP Development at Beatport – Digital download network like iTunes but specifically for DJ’s.  Several internal teams and two offshore teams in India with stakeholders in Denver, Berlin and London.

20 months into adoption – “Many great changes, but far from complete with our adoption.”

There have been four clear “states” of our Agile adoption so far.  (Cool state transition diagram of these states – If you read this blog often, you know how much I love state transition diagrams for Agile adoption.  It is really about taking incremental moves to change the engines while the airplane is in air!)

State 1 – Iterations and weak backlog. Just jumped in, but we did not whip our quality issues and 3 week release process.

State 2 – Discovered the Product Owner role and the value of meetings when way up (CEO and CTO are both DJ’s and act as customers).  “We understood the need for well elaborated stories and acceptance criteria.”

State 3 – Got the stakeholder more involved and made the backlog more clear and stopped over-elaborating far off stories.  “The best part is the collaboration and negotiation went way up!” The art of software was starting to appear and drive trust across the company.

First two states were focused on the software team, but now states 3 and 4 are focused on the larger team and company.

State 4 – Integrating vision and release planning to drive enterprise architecture.  Moved Research into Development and got IT/Operations team in synchronization with the development release cycles. Huge increase in throughput of new working code to market.

See what State 5 of the transition is but have not started this.  All focused on enterprise architecture in state 4 to manage all these moving pieces and keeping them aligned with the business.  Don’t confuse the Agile development process with the architectural process.

Clear call to actions:

1. Keep your release small and focused
2. Hired a very experienced Agile product owner
3. Have your architecture “live with your team”
4. Planning takes time, but it critical for alignment

Peggy Reed – Director of Performance Solutions at Avaya – 30 years of experience in software started in 1979 in writing Motorola code

Beautiful Software – Readability, Simplicity, Functional Locality, Single Function, System Minimality, Appropriate Form, Cooperative

NOT DOES IT WORK! – Is it pleasant, comfortable, seductive?  It has to be something a customer wants to use!

Beautiful teams make beautiful software – Needs shared vision, deep understanding, harmony in the Scrum, empathy of the customer perspective.

Beautiful teams must – Keep the base stable through Agile development of frequent public builds that integrate across the whole.  Have to always be testing with an equal balance.  Completion is a measured by ALL parts of the team.  Sometimes this is hard for existing teams to break the old habits.

Beautiful software teams talk a ton about their failures.  What we wished we had done. Talk about dependencies on each other.  Talk about feature-itis and which features really drive revenue.  Talk about teamwork and team awareness.  No longer software development by  solos and prayer.

Value of us comes from creating beautiful teams that create beautiful software in a creative way – a way that can not be outsourced or automated!

Peter Fisher – MIS Product Manager at eCollege – Working on Scrum for the last two years Pearson eCollege online learning with teams in Denver and Sri Lanka

Were a very large waterfall team with huge business requirements document – Informal adoption started in 2007 and formal training started in mid 2008.  Four releases since mid-2008 versus 1 and no more “bug builds.” Have been able to build the quality in from day 1.

Now developers are focused on building what they need to meet the tests and not their own vision.  Key to this was delivering to the integration server continuously.  As a result, the “come backs” of release went way down.  Only released to staging and production once.

This dramatically increased the speed of the team and clear visibility has allowed us to react at each two week iteration with strict prioritization.  As a result, were not delivering the wrong or unwanted features.  Throughput is going up.

Israel Gat – on his adoption experience at BMC Software

Four simple principles make the secret sauce of enterprise scale adoption at BMC.  It is all about mindset and not the practices.

In year 1 from zero to 350 scrumers in Rally at BMC.  He now showed the Cutter Consortium / QSMA slide of BMC Productivity Index of 27 versus an average of 6 to 16 in other software types.

“How many of your teams come to work on Monday determined to produce poor quality software?”  Agile teams do not have this mindset.

Principle 1 – Leadership and social contract with the team.

Principle 2 – Know How – I used Rally with a 50/50 spend on professional services and coaching while the other 50% was used for Rally’s application – this combination was critical to go as fast as we did.

Principle 3 – Flexibility - You have to take the adoption incrementally and iteratively.

Principle 4 – Patience - You can not Agile the Agile – you are changing the software, the process and the organizational structure.

More to follow this afternoon from the event…

On Thursday, March 19th, Dave West from Forrester Research and I will be doing a live webinar titled, “Scaling Agility with Lean: Proven Methods of Operational Efficiency.” It is part of a webinar series regarding “Using Agile to Cut Development Costs.” Our own Jean Tabaka and Global Logic’s Johnny Scarborough did the first webinar titled “Realizing the Promise of Agile: Creating Leaner, Meaner and Faster Product Development” and it is available via recording by registering at this same site.

In this week’s webinar, I am going to share a set of models that I  keep being asked about. These slides show the changes that happen in the enterprise stage gate model as your Agile adoption moves up and matures in the organization. To hear the explanation behind these slides and see Dave West’s work on Lean, please join us on the webinar.  The full slide deck will be available after the webinar.

Not to steal all of my thunder, but there are two things that interesting about this transition.

1. The Agile stage gates of 3.1/4.1 actually provide more governance control than the old gates.
2. The first and last gate are not going away in large organizations anytime soon. You just don’t start projects without budget and you do not make a gold master for production without blessing from operations or overall system testing.

Lastly, Dave’s observations about how these stage gate models require a Lean organization approach, should make for a great conversation. I hope you will join us.

I’ve been talking with colleague and friend Israel Gat about the upcoming Rally Success Tour March 18 in Denver. In anticipation of the event, Israel has been talking with some of the people to be involved in both a panel discussion and breakout sessions. One of the panelists, Peggy Reed, R&D Director of Performance Solutions with Avaya, is someone I have known for several years.  With 30 years in the software industry, Peggy has expertise in many areas of software development. This includes enterprise integration, data visualization, data warehousing, and product development. She began her career in 1979 writing Motorola assembly language and has since moved into development management. She holds a degree in Computer Science from Colorado State University.

I know of Peggy’s passion around organizations, teams, software, and Agile in particular. She holds a conviction that this Agile forward thinking is the true foundation of delivering better software faster.  So, it was great to hear from Israel that Peggy spoke with him about something she calls “Beautiful Software.” I had an immediate warm reaction to this! Only Peggy could come up with such an intriguing notion about software.

In this post, I interview Israel about that very enticing notion that Peggy mentioned to find out his reactions and experiences in the world of “Beautiful Software.” We promise to not giveaway any spoilers from the upcoming event. We just want to poke around what  ideas come to us with the  term “Beautiful Software.”

Jean: Israel, thanks for joining me here. First of all, could you tell us a little about yourself?

Israel: I was born and raised in Israel. I served in the paratroop corps there and fought in the ’67 and ’73 wars. I came to the United States in ’76 and have lived here for most of the time since. I thought it would be good when I came in ’76, but little had I realized then how good it would actually be. I am very thankful to the country and the people for helping me make the United States my home. I actually feel that by sharing my Agile expertise I pay back my “debt” to the United States and to its software industry. For the better part of my professional career I have been doing system management. Since I got exposed to Agile in 2003, my passion for it has been steadily growing. About two years ago my passion for Agile had surpassed my passion for system management.  As of the beginning of this year I am devoting all my cycles to Agile, particularly with respect to consulting and coaching on enterprise level Agile deployments.

Jean: Great, Israel. Can you tell me about some of what struck you most about your interview with Peggy?

Israel: The PASSION. While I could not see her body language (this was a telephone interview) her passion for software was radiating through the receiver. I felt privileged and honored to speak with a professional who put so much of both her mind and her heart to software.

Jean: Thinking about Peggy’s passions and then about your own experience in the world of software, how do you think this term “Beautiful” applies? I mean, isn’t it a rather provocative and unusual description to ascribe to software?

Israel: I had a revelation many years ago when I first read the Cluetrain Manifesto: The End of Business as Usual. I realized then that I was a craftsman, not an assembly line worker. A true craftsman is proud of the intricacies of his/her work, striving for excellence and elegance while respecting the nature of the medium with which he or she is working. To borrow the title of a recent book, a software craftsman dreams in code. This sense of great discovery I had savored so long ago (while reading the Cluetrain Manifesto) was rekindled in me while I was listening to Peggy.

Jean: I don’t want to give away too much here about what Peggy will bring us next week. Still, can you tell me where, in your experience, a notion of “Beautiful” impacts organizational structure and business motivators?

Israel: Jim Highsmith talks of Intrinsic Quality versus Extrinsic Quality. Peggy’s notion of the beauty of the software is in some ways conceptually similar to Jim’s Intrinsic Quality. It is like the visceral feeling you get while holding an iPod in the palm of your hand – you know you are touching something very special!  I will restrain myself not to get deeper on the subject here as I might unintentionally steal some of Peggy’s thunder. I definitely plan to dive deep into Beautiful Software → Intrinsic Quality topic during the March 18 event in Denver. Stay tuned…

Jean: Well, thanks Israel for giving us some of your reactions and feedback. For people in the Denver area, I encourage you to come meet Peggy and the other panelists yourself March 18 for more about Agile adoption, Agile practices, and yes, even “Beautiful Software.”  Meanwhile, Israel and I will check-in here again after the event to report how others learned about and reacted to “Beautiful Software.” See you then!

Part 2 in our discussion on successfully rolling-out Agile practices also comes from our Austin event where Jack  Yang from Homeaway spoke about a team-by-team approach. (See part 1 for more background on the Austin event and Israel Gat’s post on the Agile roll-out at BMC.) You can attend an event like this yourself in Denver Mar 18, Los Angeles Mar 26 or New York City Apr 2. It’s free, but you do need to register on our web site.

Please chime in with your success stories, or horror stories, on Agile roll-outs.

Implementing Agile Principles – Trials and Tribulations of a Team-by-Team Approach, by Jack Yang

I’ve had the privilege helping to roll out Agile/Scrum adoption here at Homeaway and a few other companies. At HomeAway, the process of adoption wasn’t simple – we faced a series of obstacles and false-start attempts until achieving a mature company-wide development process. Throughout the technology world there are many proponents for and against “all in” or team-by-team approach. In retrospect, given the challenges we faced as a company, the team-by-team approach was ultimately the most successful for us. I have no doubt that, given a different set of starting parameters, a large full-scale implementation throughout an organization could be successful. My belief, however, is that most teams aspiring towards an Agile process (which ever flavor it may be), start with a grass roots movement that requires careful proof of value. This may preclude the ability for a large rollout. To get a better understanding on how the HomeAway development team rolled out Agile, it is important to understand the environment of the company and how we, as an organization, happened upon trying Agile and the various barriers we faced, and continue to face, in becoming and staying Agile.

Homeaway.com, was launched in June 2006 to consolidate a cottage industry of vacation rentals and transform the industry into competitive alternative towards traditional vacationing. It is important to note that HomeAway is an aggregate of five smaller companies. The consolidation of HomeAway’s smaller companies did not happen over several years, which may have allowed for careful absorption of technology and staff. Instead, it started with a large buy-up that continued over the next three years as we continued rapid growth through acquisition. Each company brought new technologies, differing levels of development experience, and cultural backgrounds. Business units that were formerly autonomous were now required to play in a larger field that had differing often competing interests. The headquarters suddenly became the “mothership.” Satellite offices were cautious of change and each of the formerly autonomous businesses were highly profitable and had been operating that way for some time. Ultimately, however, we needed to create a consolidated brand. That meant everyone had to effectively work together. The company’s executive team focused on getting the each brand (or business) running in the same direction. This meant the development staff was now committed to many releases and ad-hoc requirements that changed often

The early days at Homeaway were challenging. My team at the time consisted of four engineers and when they tired of the poor release code quality, a brittle production environment, and long deployment times (at times, it took the team up to nine hours to finally roll out to production), we were driven to come together and commit to a different process. The team was made up of several seasoned startup veterans and a few developers new to the startup environment. We decided as a group that our ad-hoc “Agile / Waterfall” process did not work. Instead, we needed to pick one methodology and stick with it. We began with the following mission, “Follow the book to the letter to start, slow down to one single story and get it done with quality” and a philosophy that our one QA team member was the most important person on our team. I had the fortune of having a Product Owner that understood and shared in my team’s frustrations, so I had buyoff.

Within a couple of months, we were one of the most productive teams in the organization, producing quality code that carried less than 10 defects from sprint-to-sprint. Going Agile was a big part of the success, while facilitating adoption and enforcing of best practices was the other piece. Our particular product didn’t generate the most revenue for the company, but the sudden boost in speed quickly drew attention from senior management.

At this point, the management team decided to split this team and reseed the rest of the development organization with Agile members to help spread out the knowledge all at once. We hired a coach who focused on certifications for a few U.S. developers first, and then went overseas to train and convert our European counterparts. So, we essentially tried to rollout Agile throughout the organization in an “all in” approach. However, this failed on a global scale.

Here is why that approach did not work:

• The individual seeds had the monumental responsibility of convincing their new teams to do something new.
• Development managers felt they were being told how to run their teams.
• Product Owners were accustomed to using existing requirements binders and were unwilling to give them up.
• Product Owners lost the ability to ask developers “can you just fit that one feature in?” and felt like they lost control over their product.
• Evangelizing to an entire office that didn’t buy-in/understand the process resulted in many individuals paying lip service to the new process – “We’ll do it the old way when no one is looking.”
• Developers were used to hacking in features and deferring technological debt. They were now being asked to cleanup as they went along. In their eyes, this meant they weren’t moving as quickly as before.
• C-level executives didn’t see the same ramp up of quality and withdrew their support for Agile.

During this period, I was asked to form and staff up two new teams around key company initiatives. I was allowed two seed team members from my old team and then hired new employees. The failed attempt of Agile adoption across the organization put the newly-formed team at a heavy disadvantage when we elected to follow strict Scrum practice. However, the two new teams carried on, despite doubt and pushback from senior management and product teams.

Eventually, we launched the projects on time and with high quality. Using historical data, I was able to demonstrate costs on new projects and give visibility to team progress. This was a turning point for the company. With three teams having successfully implemented Agile, there was no question it could work and the first team’s success was no longer viewed as just a fluke. This time, with the support of the CTO, we resisted splitting the teams. And, the technological debts of “getting things out” and “not doing the right thing” caught up with the teams who were resistant to Agile practices. As each team got fed up with a lack of quality and inability to deliver, they began looking to the teams who were surpassing their goals and asking for help. This time change was socialized. More importantly, we were now rolling out teams in a methodical team-by-team approach, rather than trying to rollout all teams at the same time.

Here is my analysis of why we ultimately succeeded on this second attempt at rolling out Agile practices:

• Teams that ask for help are more likely to implement, as it is “their idea.” The best way to encourage teams to ask for assistance is to show them how much better it can be.
• Due to lower technology debt overhead, professionalism and disciplined work over the medium to long term will outpace “cowboy startup” teams.
• C-level management responds to hard data and winning their support the second time around was critical to our ability to implement and manage the process across our global offices.
• All Agile methodologies require discipline. From discipline comes professionalism and teams saw they were being treated as professionals instead of hackers.
• Professional teams developed a high degree of trust, making them even more effective and eliminating egos.
• As more teams come online the process of conversion accelerates.

In summary, I believe that most companies have a very traditional core belief system and value waterfall development. They tend to be afraid or not educated on the benefits of moving to an Agile philosophy. If this is true, the only way to spread Agile would be one of two ways:

1. Either through a team-by-team, grassroots approach as demonstrated above
2. Rollout Agile from the top down, with executive team support

Executive teams tend to be conservative when it comes to changing their core beliefs and will naturally wish to only “pilot” a team to begin. This obviously leads to a team-by-team approach. My view is the amount of change that needs to happen to perform an “all in” rollout within a large organization is riskier than the same approach used with a small company. The odds of failure anywhere in the system are high. However, when driving adoption, the consequences of failure were so high that we didn’t feel like we could risk an “all in” approach as we feared we would be in a worse place than when we had first initiated the change.

Original F4 Technologies logo 2002-2004

I know what you are thinking with the title of this post – I am drinking the Kool-Aid. Just bear with me  for a minute. Back in 2002, when I started working on Rally, it was originally known as F4 Technologies. It was known as F4 because I did not want to work on anything that did not have the potential impact of a Factor of Four, for example a 4X increase in productivity or effectiveness.   There are two reasons for this:

1. Andrew MacAfee and John Gourville at HBS have shown that you need a 9X improvement with a new tool, technique or method to un-seed the incumbent.
2. According to Paul Hawken in Natural Capitalism, we need a 4X productivity increase in the use of natural resources to get to a sustainable place with the current population in the world.  Chapter 7 of that book helped form the core purpose and mantra of Rally – “Muda, Service and Flow.”

Now seven years into Rally, we have the proof that teams – including large and distributed teams – can be 4 to 10X more productive by following Lean principles and effectively implementing Agile development.  Like the story of “Good to Great”  from Jim Collins, you can’t leap here, but you can put yourself on that path by adopting a continuous improvement approach like Agile.  If you do that, you can be a “great” software development organization that dominates your market and is 10X better than the good ones.  Great organizations that dominate in their industry also have the knowledge and resources to change world, a la Google.org, the Salesforce.com Foundation and or through my favorite the Entrepreneur’s Foundation.

My summary take-away from Good to Great is:

“Right people building the right things right”

“Disciplined people, Disciplined thought, Disciplined culture “

If you are working toward this, I believe you increase your business value by 4 to 10 times.  I am going to make the case with the help of ROI models from David Anderson.  (BTW, I love his book – it does a great job explaining the simple physics of Agile.)

From Chapter 2 - David Anderson's "Agile Management for Software Engineering"

This is a very simple model of software process.  David shows more complex ones that model all the loop backs of large shipping software, but let’s work with this one.  So, the rough equations to calculate the business benefit of the process are the following:

Net Profit = Throughput – Operating Expense 
ROI = Net Profit / Investment

In the following four pages, I am going to look at how this equation plays out for four different scenarios:

1. Good waterfall team, on the mean line of the QSMA Agile Impact Report
2. Beginning Agile team in Flow benefiting from the 25% productivity savings of an Agile teams in the same study
3. Intermediate Agile team in Pull with incremental releases of value
4. Advanced team in Innovate that cuts time-to-market in 1/2 to end early after delivering 50% of the work but 80%the value

What you will see in this hypothetical modeling exercise is the true power of Agile to dramatically impact the software development teams in the organization. For a deeper understanding of what I mean by Flow, Pull and Innovate, please Jean and I’s white paper on moving to Program Pull.

Here is the summary:

1. Good waterfall team – ROI – 0.8
2. Beginning Agile team in Flow  – ROI – 1.4 (1.6 factor better than good waterfall team)
3. Intermediate Agile team in Pull  – ROI – 2.6 (3.2 factor better than good waterfall team)
4. Advanced Agile team in Innovate  – ROI – 6.3 (7.7 factor better than good waterfall team)

Factor or Four or better – that is why there is such a rush towards Agile development.  Of course, you can’t have your cake and  eat it too.  Moving up this maturity curve takes long-term dedication to increasing discipline and agility across the entire organization, but there are dramatic benefits if you can get on the continuous improvement path and stay there.

Late last year, we held an event in Austin where four customers spoke about their experiences with Agile development.

One of the big discussions that we had centered around how to successfully roll-out Agile across a team or organization. In the posts that follow, authored by several speakers from the event, you’ll find some different takes on Agile roll-outs. I’ve also posted video segments from the event.

We launched that pilot event into a nationwide tour in 2009, of which the first stops are Denver Mar 18, Los Angeles Mar 26 and New York City Apr 2. Depending on your city, speakers are AOL, Avaya, Beatport, BMC, Condé Nast Digital, eCollege, Gorilla Nation, MPower IT, NBC Universal, Qualcomm, Real Capital Markets and Spatial.

It’s free, but you do need to register on our web site.

(UPDATE : new event dates are being added regularly and can be found here)

How did you roll-out Agile development – team-by-team, all-in, another way? And what did you learn from the experience?

Rolling Out Agile Team-by-Team, by Israel Gat

Jean in the mini-kitchen at Rally headquarters.

Jean has been doing a series of posts (What is with all the Agile process overhead? – part 1,  What is with all the Agile process overhead? – part 2, and How do you plan for unplanned work? – part 3.) on what goes in your backlog.  We have both noticed that it is really tempting to put the kitchen sink in your backlog.  At Rally, we only do progressive elaboration of stories based on subsequent levels of planning, which ends up looking a bit like this:

• Ideas and research topics managed by product line on the wall and adjusted across product lines every quarter
• Epics at the roadmap level (multiple release level) managed in a roll-up project in Rally’s Agile project management solution and estimated every eight weeks for release planning
• Stories at the release level managed in individual projects in Rally elaborated every eight weeks at release planning
• Tasks at iteration level managed by team member in Rally and elaborated every two weeks at iteration planning

As a result, we keep our backlog well groomed and do not waste time (muda) managing excess inventory or over-investing in stories that are not guaranteed to get built.  I have noticed it is really tempting to try and load all your ideas, suggestions and half-baked stories into a product like Rally.  We built Rally to make that uncomfortable – to try and discourage that.  Rally focuses you and the team on the iteration and not a big list of defects and bugs, while our coaches and technical account managers try to help folks move away from thinking of our system as an inventory manager.  We think of Rally and design it as a real-time decision support system.  (See my 2006 article in SD Times on “Kill your inventory manager” for more on this topic.)

As Jean and I were talking about this in our kitchen, I was noticing something  (see the picture). In this picture, there are no dirty dishes in the sink.  This is a relatively new occurrence here at Rally.  It happened when we moved from our old location, which had a huge kitchen and two dishwashers, to this new building. We doubled our square feet but shrunk our kitchen space in half with no dishwashers as a result of an in-house cafeteria that we share with Oracle and Quantum.

With almost no counter space and no dishwasher to batch up dirty dishes, you have to clean your dishes as you use them.  For the 150+ people who work at this office, that means a real just-in-time process of dirty it and then clean it.  As a result, we have less dishes, less wasted time and less wasted space due to managing the dirty dishes.  We also lost the typical sign that says, “Your mother does not work here! Please clean-up after yourself.”

Excess inventory leads to wasted time, wasted effort, and friction.  If you keep your backlog well groomed and focused on the valuable items, as Jean illustrates, you can keep from wasting valuable time.

This post is in honor of Greg Macaluso and Kevin Mindenhall, both manufacturing and distribution process consultants from Coopers & Lybrand.  In 1993, they taught me the way of JIT and Lean while working on an MRP project at Robinson Brick and a remittance processing project at U S WEST Communications.  In addition to tutalage, they had me read and flip the wonderful pictures of JIT, kanbans and manufacturing facilities with no horizontal space.  By removing the horizontal space to store stuff, the raw and work-in-process inventory went down and the throughput went up!

Team 100% allocated commits to 8 Items for the Iteration

This is the final post in my series about a client’s consternation with regard to Agile Process Overhead. This last topic had to do with their Sprint Planning Meeting and how to make commitments.

The problem? “Jean, how do we plan for unplanned work?”

In this case, the team was referring to production issues that would come up that HAD to be addressed, even though they could not be predicted during the Sprint/Iteration Planning Meeting.

So, how DO you plan for unplanned work? If you look at my post about the 5 Orders of Ignorance, where would you place this issue? Well, it is the 1st  Order of Ignorance:

• 1st Order Ignorance: Lack of Knowledge
  • I do not know something

For this team, in their planning meeting, they truly can say with conviction, “We know that we don’t know something.” That is, for this team, they know that they can’t know all the production issues that may come up. Fortunately for this team, they don’t fall into the higher orders of ignorance. Most notably, they are not in the clammy grasp of the 3rd Order of Ignorance:

• 3rd Order Ignorance: Lack of Process
  • I do not know a (suitably effective) way to find out that I don’t know something

We have a process! It is called Agile software development! And here is how Agile helps us plan for unplanned work, the things we know that we do not know.

1. Do not commit all of your team capacity to the work you DO know about (i.e. Prioritized Backlog Items)
2. Track Iteration over Iteration the amount of unplanned production work that comes in (e.g. “Last Iteration we didn’t finish 3 of our committed items because we had 5 major production issues come in. The Iteration before that, we had 6 minor production issues come in; we did not complete 1 of our committed items.”)
3. Declare and hold a buffer that is an average percentage of time you have production/unplanned work arrive after Sprint Planning (Iteration = Sprint in Scrum) has completed (e.g. “Based on the last 5 Iterations, we have spent an average of 30% of our time on production issues. Therefore, we must hold back a 30% buffer from our Sprint Planning capacity this timebox.”)
4. During the Sprint/Iteration, continue to track how much unplanned work comes in that impacts the team.
5. Report this unplanned work and its impact in the Sprint Demo and Review meeting. Continue to evaluate its impact on the team’s ability to take on new, value-based, innovative work during the Sprint/Iteration.
6. If NO unplanned work comes in during the Sprint, consult with the Product Owner about the next highest priority items the team could invite into their commitment for the Sprint. (Some teams like to talk about this “back burner” work during the Sprint Planning Meeting. They plan it out with tasks and estimates but do not commit to it.)

Team buffers and only commits to 6 Items for the Iteration

Finally, a team that finds itself continually taking on more and more unplanned work needs to run a Retrospective specifically on this topic.

This is part and parcel of a real Agile culture: retrospect on items that are getting in the way of our ability to deliver value.

Where is all the unplanned work coming from? Do we really not know our priorities? Do we really not stick to our Sprint Plan?

Is our product technical debt so great that we are forever mired down in triaging the product? Is our testing strategy not disciplined enough with regard to how we declare “Done” items? What new agreements or strategies must we put into place to reduce the amount of unplanned work?

Now we have a process for how we deal with work that we cannot plan for. Hold your resolve around this. Do not be pressured into fully filling your Sprint/Iteration until you are able to successfully deliver committed items.

Your Burndown Chart will have a more gentle slope; your Sprint/Iteration backlog will have fewer items. AND, you will be able to absorb that work we know that we don’t know.

About the Author: Jean Tabaka is a wine enthusiast, author and Agile Fellow at Rally Software Development. Subscribe today to get free updates by email or RSS.

Before I dive into Part 3 on my series about Agile Process Overhead, I wanted to review where we are and interject a little wisdom around all of this.

You may recall that I had a client come to me with great concern (frustration?) about all the “overhead” in adopting Agile. Specifically, they wanted to address:

• What goes in the Sprint Backlog?
• How do you allocate your time effectively?
• How do you “plan” for unplanned work?

Each of these seemed to cause a lot of overhead for their ScrumMaster and the team. That confused me and so I decided to post some ideas about them. I addressed the first bullet in Part 1 of this series, and the second bullet in Part 2.

So let’s take a break before moving on to Part 3,: “How do you ‘plan’ for unplanned work?” Here is what I want to raise as a guiding mental model: the 5 Orders of Ignorance. This may appear to have nothing to do with the perceived process overhead with Agile. I think it has everything to do with it.

Where do the 5 Orders of Ignorance come from? Phillip G. Armour outlined these in his book The Laws of Software Process. (You can also find a quick outline of them in a 2000 Communications of the ACM Article.)

If we are looking at process overhead, I think applying the 5 Orders of Ignorance is great guidance:

• 0th Order Ignorance: Lack of Ignorance
  • I (probably) know something
• 1st Order Ignorance: Lack of Knowledge
  • I do not know something
• 2nd Order Ignorance: Lack of Awareness
  • I do not know that I do not know something
• 3rd Order Ignorance: Lack of Process
  • I do not know a (suitably effective) way to find out that I don’t know something
• 4th Order Ignorance: Meta-Ignorance
  • I do not know about the 5 Orders of Ignorance

Where do you think the troubled team is in the 5 Orders of Ignorance when they are struggling with Agile Process Overhead in the guise of an overloaded Sprint Backlog, allocating team members, and planning for unplanned work? How do you think Agile and Lean approaches actually directly impact the 5 Orders of Ignorance? And, in so doing, can you see ways that, in attacking the 5 Orders of Ignorance, they directly impact costs to teams and organizations in a variety of domains?

Okay, next post I will complete Part 3 of our Agile Process Overhead series. And we’ll find out how Agile addresses these 5 Orders of Ignorance. Stay tuned!

About the Author: Jean Tabaka is a wine enthusiast, author and Agile Fellow at Rally Software Development. Subscribe today to get free updates by email or RSS.