Agile Development Blog

Last Thursday, Dave West from Forrester and I did a live webinar on Scaling Agility with Lean: Proven Methods of Operational Efficiency.

We received lots of great questions and shared them with the webinar attendees, but thought they would also be valuable to other Agile practitioners. This first series of questions tackle the obstacles of adopting Agile. Stay tuned for the remaining questions in later posts.

You can also check out the first webinar, hosted by Jean Tabaka and Johnny Scarborough from Global Logic, on Realizing the Promise of Agile: Creating Leaner, Meaner and Faster Product Development.

DW – is Dave West from Forrester

RAM – is Ryan Martens from Rally

What are the biggest blockers you see to good Agile?

DW – My pet peeves are people that resist change without really thinking about it. That treat software like car production and think that you can plan, deliver in that manner. These same people often lament the ability of the software teams to deliver good code at the right price. Software is a fundamental part of business change – If you need a new part to your business it will most likely require software, which may require development and integration. Connecting up their development with the people who have the problem is the first step to success, empowering teams to try new things and question the status quo in support of those business needs is key. Organizations that put artificial walls between the problem and the solution are the biggest blockers. These may come in the form of planning processes, business access, travel budgets, governance models, standards or culture.

RAM – Good Agile development is a mindset change like Dave is describing above. Overly aggressive or passive plans are the biggest blockers to “good Agile.” You need to think about this change as incremental and iterative with a strong focus on continuous improvement and learning through inspect and adapt feedback loops.

(more…)

Original F4 Technologies logo 2002-2004

I know what you are thinking with the title of this post – I am drinking the Kool-Aid. Just bear with me  for a minute. Back in 2002, when I started working on Rally, it was originally known as F4 Technologies. It was known as F4 because I did not want to work on anything that did not have the potential impact of a Factor of Four, for example a 4X increase in productivity or effectiveness.   There are two reasons for this:

1. Andrew MacAfee and John Gourville at HBS have shown that you need a 9X improvement with a new tool, technique or method to un-seed the incumbent.
2. According to Paul Hawken in Natural Capitalism, we need a 4X productivity increase in the use of natural resources to get to a sustainable place with the current population in the world.  Chapter 7 of that book helped form the core purpose and mantra of Rally – “Muda, Service and Flow.”

Now seven years into Rally, we have the proof that teams – including large and distributed teams – can be 4 to 10X more productive by following Lean principles and effectively implementing Agile development.  Like the story of “Good to Great”  from Jim Collins, you can’t leap here, but you can put yourself on that path by adopting a continuous improvement approach like Agile.  If you do that, you can be a “great” software development organization that dominates your market and is 10X better than the good ones.  Great organizations that dominate in their industry also have the knowledge and resources to change world, a la Google.org, the Salesforce.com Foundation and or through my favorite the Entrepreneur’s Foundation.

My summary take-away from Good to Great is:

“Right people building the right things right”

“Disciplined people, Disciplined thought, Disciplined culture “

If you are working toward this, I believe you increase your business value by 4 to 10 times.  I am going to make the case with the help of ROI models from David Anderson.  (BTW, I love his book – it does a great job explaining the simple physics of Agile.)

From Chapter 2 - David Anderson's "Agile Management for Software Engineering"

This is a very simple model of software process.  David shows more complex ones that model all the loop backs of large shipping software, but let’s work with this one.  So, the rough equations to calculate the business benefit of the process are the following:

Net Profit = Throughput – Operating Expense 
ROI = Net Profit / Investment

In the following four pages, I am going to look at how this equation plays out for four different scenarios:

1. Good waterfall team, on the mean line of the QSMA Agile Impact Report
2. Beginning Agile team in Flow benefiting from the 25% productivity savings of an Agile teams in the same study
3. Intermediate Agile team in Pull with incremental releases of value
4. Advanced team in Innovate that cuts time-to-market in 1/2 to end early after delivering 50% of the work but 80%the value

What you will see in this hypothetical modeling exercise is the true power of Agile to dramatically impact the software development teams in the organization. For a deeper understanding of what I mean by Flow, Pull and Innovate, please Jean and I’s white paper on moving to Program Pull.

Here is the summary:

1. Good waterfall team – ROI – 0.8
2. Beginning Agile team in Flow  – ROI – 1.4 (1.6 factor better than good waterfall team)
3. Intermediate Agile team in Pull  – ROI – 2.6 (3.2 factor better than good waterfall team)
4. Advanced Agile team in Innovate  – ROI – 6.3 (7.7 factor better than good waterfall team)

Factor or Four or better – that is why there is such a rush towards Agile development.  Of course, you can’t have your cake and  eat it too.  Moving up this maturity curve takes long-term dedication to increasing discipline and agility across the entire organization, but there are dramatic benefits if you can get on the continuous improvement path and stay there.

Jean in the mini-kitchen at Rally headquarters.

Jean has been doing a series of posts (What is with all the Agile process overhead? – part 1,  What is with all the Agile process overhead? – part 2, and How do you plan for unplanned work? – part 3.) on what goes in your backlog.  We have both noticed that it is really tempting to put the kitchen sink in your backlog.  At Rally, we only do progressive elaboration of stories based on subsequent levels of planning, which ends up looking a bit like this:

• Ideas and research topics managed by product line on the wall and adjusted across product lines every quarter
• Epics at the roadmap level (multiple release level) managed in a roll-up project in Rally’s Agile project management solution and estimated every eight weeks for release planning
• Stories at the release level managed in individual projects in Rally elaborated every eight weeks at release planning
• Tasks at iteration level managed by team member in Rally and elaborated every two weeks at iteration planning

As a result, we keep our backlog well groomed and do not waste time (muda) managing excess inventory or over-investing in stories that are not guaranteed to get built.  I have noticed it is really tempting to try and load all your ideas, suggestions and half-baked stories into a product like Rally.  We built Rally to make that uncomfortable – to try and discourage that.  Rally focuses you and the team on the iteration and not a big list of defects and bugs, while our coaches and technical account managers try to help folks move away from thinking of our system as an inventory manager.  We think of Rally and design it as a real-time decision support system.  (See my 2006 article in SD Times on “Kill your inventory manager” for more on this topic.)

As Jean and I were talking about this in our kitchen, I was noticing something  (see the picture). In this picture, there are no dirty dishes in the sink.  This is a relatively new occurrence here at Rally.  It happened when we moved from our old location, which had a huge kitchen and two dishwashers, to this new building. We doubled our square feet but shrunk our kitchen space in half with no dishwashers as a result of an in-house cafeteria that we share with Oracle and Quantum.

With almost no counter space and no dishwasher to batch up dirty dishes, you have to clean your dishes as you use them.  For the 150+ people who work at this office, that means a real just-in-time process of dirty it and then clean it.  As a result, we have less dishes, less wasted time and less wasted space due to managing the dirty dishes.  We also lost the typical sign that says, “Your mother does not work here! Please clean-up after yourself.”

Excess inventory leads to wasted time, wasted effort, and friction.  If you keep your backlog well groomed and focused on the valuable items, as Jean illustrates, you can keep from wasting valuable time.

This post is in honor of Greg Macaluso and Kevin Mindenhall, both manufacturing and distribution process consultants from Coopers & Lybrand.  In 1993, they taught me the way of JIT and Lean while working on an MRP project at Robinson Brick and a remittance processing project at U S WEST Communications.  In addition to tutalage, they had me read and flip the wonderful pictures of JIT, kanbans and manufacturing facilities with no horizontal space.  By removing the horizontal space to store stuff, the raw and work-in-process inventory went down and the throughput went up!

If you have ever wondered what to do with the growing number of features in your software applications, you are not the only one. There is some useful guidance from The Standish Group survey quantifing that only 20% of all delivered software features are “often or always used.” For a team under the pressure of today’s economy to do more with less, here is a way to conceptually cut 80% of your effort and still deliver the high-value essentials. But wait it gets better. If you do not completely elaborate, build, test or document these features, you save once. But you also save again: with the future maintenance costs, the training users, the bug fixing and the regressing these features would have required. This is clearly the biggest lever for cutting software development costs using Agile. And yet it seems to be the hardest to attain.

Features used in Commercial Software - Reported by Jim Johnson of the Standish Group at XP2002

This pairing back to the simple, important few takes tons of discipline and that level of discipline is only be made easier as your agility maturity increases. Most agile books and coaches will call this the discipline of “Brutal Prioritization.” The folks from 37Signals call it: “Getting Real.” In either case, brutal prioritization means two things:

1. Not letting the fat slip into the backlog
2. Keeping the backlog prioritized by value and risk

First, lets talk about fat or wasteful features. Please note, fat or wasteful features come from all directions from the business and customers, but also from developers, support, and product owners. Your features may be fat or wasteful if they:

1. Do not help the most important user persona in your domain achieve breakthrough results or competitive advantage
2. Have not be proven to work and are based on a hopes and prayers
3. Look like pets that someone is protecting
4. Are weak features that degrade the product because they are not complete enough to meet minimum expectations

Photo © Tristan Savatier - http://loupiote.com/ - Used by Permission

So if you are in the customer role or customer proxy role such as the Scrum product owner role, the technical product management or a business analyst role, it is your job to control yourself and help the team to keep the fat off the backlog. If you are on the agile team, it is your job to keep these roles honest about WHY these features are so valuable. This leads right to ranking the items into a prioritized backlog. (A few comments in this area would be really helpful – let’s hear what you use to spot fat features?)

(more…)

[ UPDATE:  Fall 2009 Locations have now been announced for Boston, Seattle, Chicago and London ]

In December 2008, we ran the prototype to our recently announced Agile Success Tour in Austin, Texas.  This event was extremely well received, and the majority of attendees would recommend this event to their colleagues.   Given the high satisfaction level, I can almost guarantee you will not want to miss the one nearest you.  So consider registering for upcoming events in Denver, Los Angeles or New York City.  Short of living in Chicago and attending Agile 2009 in August, it has to be the best way to learn from your local peers about the actual benefits and best practices of adopting, scaling and cutting costs with Agile software development.

I was the moderator at our Austin event and while there, we captured short videos of our customers/presenters.  These customer videos give you a sample of what you can expect to see and feel in your community.

The videos include comments from:

• Israel Gat – former VP of Development at BMC
• Jack Yang –Director of Engineering, HomeAway
• Gary Allison – VP of Engineering, Convio
• Erik Huddleston — CTO, Inovis

I really enjoyed two things about the event.  First, attendees got to hear and discuss multiple approaches to Agile adoption, tooling Agile lifecycle management and the business drivers that drove their move to Agile.  Second, the interplay between local leaders, including members of the Agile Austin community, resulted in shared learning while enhancing the local professional network.  In addition to a brief presentation by Israel, these gentlemen were all allotted enough time to share their stories, answer a few of my questions and field audience questions.  Following the panel, breakout groups allowed folks to dive deep on hot topics, get introductions to Rally partners and learn about major enhancements to Rally’s services and applications for Agile lifecycle management.  Of course the event was executed flawlessly by our great marketing team including Sonya Breakstone and Michelle Burrows, and was facilitated by Julie Chickering, our Texas-based coach.

The Denver event is set for Tuesday, March 18th in downtown Denver.   This event will include:

• Peggy Reed — VP of Development at Avaya
• Lloyd Star — VP of Engineering/Development at Beatport
• Pete Fischer — Product Manager at eCollege
• Ray Bagley — Director, Product Planning & Management at Spatial

The Los Angeles event is set for Thursday, March 26th in Manhattan Beach, CA. The event will include:

• Christophe Louvion, CTO at Gorilla Nation
• Laureen Knudsen, Sr. Dir. of Program Management at Qualcomm
• David Annis, VP of Software Development at UTI from Arizona
• Chris Babcock, VP of Technology at Real Capital Markets

The New York City event is set for Thursday, April 2nd. This event will include:

• Jochen Krebs, Dir. of Program Management at AOL
• Brian Stockmoe, Sr. Dir. of Program Management at NBC Universal
• Land du Pont, Executive Dir. of Product at Conde Nast Digital
• Micah Silverman, Founder & Principal at MPower IT

So, please consider sending a senior member or members of your team to learn from and share with your local peers.  Additionally, Israel Gat is slated to participate in each of these three events, where he will share his stories and insights from BMC and other Agile organizations.

Registration is online and is limited to the first 100 people.  I do not believe there is an event with a higher return on your personal investment for these times.  Agile is proven to dramatically cut costs and increase innovation in any sized software development team.  If you are not realizing major benefits from Agile in your organization, you will feel the pull to get started at this event.

As a warm-up to these events, you might also consider having members of your organization attend or view the recordings of a new, two-part Webinar series on  Agile Cuts Costs that includes Jean Tabaka and me.

In the mid-1990’s, I worked on a great team (which included three members of Rally’s current technical team) as a consultant and eventually as a Director of IT at US WEST Communications in a part of the IT organization that became known at the Global Village Labs.

You can read about some of our early Intranet work in this 1995 Fortune Magazine article.  In addition to being filled with great people, we had a great leader by the name of Sherman Woo (great profile on ZoomInfo), who was a complete rebel.

Sherman, a 25-year veteran of the Bell system, was rebelling against slow IT services in the age of the 1996 Telecommunications deregulation act. To meet the rapidly changing needs of open network, he built this amazingly agile organization that leveraged Mosaic and then Netscape browsers to screen scrape mainframes using ORAPerl.

We used these agile teams of 5 to 10 people to build tools for customer service representatives and field technicians.  These were typically simple tools that bridged a couple of mainframes to build compound views that could answer really tough questions.

These questions were hard because each role was only trained in a limited scope in limited mainframes and that full training took 24 months.  So we used early Internet technology to open the access and bridge that gap with simple tools.

We made it into Fortune Magazine because our time to market was measured in months and our ROI was 1000%.  In our agile model, we delivered our first working increment to the business on test servers in a fixed time box of six weeks.  If these solutions showed value and promise for 5 users, we would run another 6-week time box and then roll the application to 50 users.  Successful projects then went on to 500, 5,000 and then up to 20,000+ users following this 6-week time box approach.  Marginally valuable applications got tabled at the end of the 5 or 50 user-demonstration time-box.

As a result, we grew the group from the initial 10 folks to the 150 person organization in IT in just 18 months.  (See the internal brochure we used to drive business to our group – (BTW, this was really fun to research on the Internet – such good memories.)

QSMA Agile Impact Report showing Agile teams 25 to 50% faster versus 7,500 other projects

50% Faster Time to Market OR 50% Less Staff?

It was all about Time to Market (TTM) for GV Labs and for most folks adopting Agile in this decade.  However, now with the cost-cutting focus of these turbulent times, we see many organizations trading off some of the TTM for cost savings.  As a result, they will use fewer Agile teams on a project and deliver a lower throughput per agile, time-box cycle.  In many cases they are using fewer people, because they have fewer people due to staff cuts.  (If you have not taken staff cuts and are wondering how to do that and maximize your Agile success, read an earlier article of mine on TechTarget – Cutting Your Way to Increased Agility and my post on Israel Gat’s Social Contract for Agile Software Development)

Of course this is a continuum of speed and these savings exist in either getting done faster with more or getting done slower with less.  You and your organization should be careful to consider the total cost and benefit of the project when right-sizing the team size to completion date. There is an optimal range for most projects and with true agile project teams you get the benefit of adjusting this based on actual performance over short time-boxes.

To gain these benefits, you have to actually become Agile and not just adopt some of the practices. Agile means dedicating you and your team to learning and continuous improvement.  We did it at US WEST in the 90’s and QSMA shows how many of our customers are doing it today.

If you truely become agile in these times, you can accelerate out of this downtrun smarter, stronger and leaner.

Further Reading:

• Brutal Prioritization in Agile: cut costs by NOT building the fluff
• Agile Cuts Costs by a  Factor of Four – or More!
• How Does Agile Deliver Time to Market Improvements of 50%?

In this 20-minute interview with Michael Vizard, I do a wide survey of why Agile development practices can be so effective at cutting the costs of development and operation of strategic IT applications. Michael asked some great questions, including whether the current economic recession will stimulate change in the development process or cause it to stagnate, how people will train and staff for the coming changes, and how today’s ALM tools differ from what he refers to as the “old guard,” who are starting to claim that their tools also support Agile development.

Read related posts on:

Can you afford the software you are developing? (from Israel Gat’s blog)

Cutting costs through productivity improvements, and

WHY do you develop and operate software?

Let me know what you think!

Back in 2007, Jean and I developed an evolutionary model of Agile adoption for teams and organizations seeking the benefits of “scaling software agility.”  We presented talks on this at the Agile 2007 and Agile 2008 conferences.  We call it Flow-Pull-Innovate based on the Toyota Lean principles of Flow-Pull-Perfect.

Productivity gains in Agile development teams come conceptually from eliminating waste.  Just getting to the first step of Agile maturity can lead to 10 to 20% productivity increases.  We want you to be successful taking the first step, and there is a ton of opportunity in most software development shops. According to Tom and Mary Poppendieck, most software development organizations are only spending 6% of their time doing value-added tasks.  The other time is wasted in these categories:

• Partially done work (coding features that get removed from release)
• Extra processes (elaborating requirements that do not get built)
• Extra features (features that are rarely used – more on this in another post)
• Task switching (across multiple projects – losing flow)
• Waiting (requirements, designs, feedback, builds, other teams, larger organization, customer)
• Motion (walking over to interrupt folks for a build or status)
• Defect (internal or external)

Agile works to systematically address these wastes as you mature.  In our white paper on moving to Program Pull, Jean and I characterize the steps, roadblocks and benefits found at each step in the Flow-Pull-Innovate maturation process.

On the move to Flow, we find that teams can increase their productivity working as a dedicated team on a single project for a two-week iteration cycle.  The reduction in waiting, motion, task switching and partially done work can be dramatic based on your level of multi-tasking.  We even find some teams in Flow reduce their extra features, partially done work and defects, but those productivity gains tend to be more associated with the move to Agile Pull.

According to the QSM Associates study of Agile development teams, you can expect to save 16% (the green dots on the graph). Of course the teams that worked with Rally’s products and services achieved on average 25% productivity gains (the red dots on the graph).

(The graph on the right from QSMA shows a white line of the average productivity of different sized projects in their 7,500+ project database. Between the blue dotted lines is the area 1 standard deviation from the norm.  You can see only one Agile project was less productive that the average and  7 Agile projects of the total 29 were outside 1 standard deviation. Thus 1/4 of these Agile projects were in the top 16% of all projects ever benchmarked by QSMA – see Michael Mah’s blog for more discussion – OptimalFriction)

Of course,  you cannot have your cake and eat it at the same time.  That is, you cannot just start running two-week iterations and become 20% more productive overnight, but with a combination of services and Agile project management solutions, you can attain these results in months. That is why we recommend a two-pronged approach to cutting costs with Agile and Rally offers a Guarenteed Succes program that combines our world class services and application in a discounted bundle for new and existing customers.  Get started by getting your teams to Flow and the savings can come fast.

I have a friend who was my Engineering Geology Professor and his name is Bernard Amadei.  Bernard found his “good work” right about the time I was starting Rally.  Bernard has a wonderfully friendly manner and a great French accent.

In early 2000’s, he took a trip to Mail to help a village engineer a solution for potable water.  Upon his return, he started working on Engineers Without Borders, which is now a movement.  At the beginning of this work, I collaborated with Bernard as he started the chapter at the University of Colorado and wove it into Bioneers.

At that initial collaboration, Bernard said he was forcing engineers to ask the question, Why? So often engineers just focus on how and what.  He was starting Engineers Without Borders to help give engineers a stronger sense of purpose in solving the world’s systemic problems. (Please see the YouTube video)

Bernard and Paul Hawken had a huge influence on me as I worked to draft the vision for Rally. You can read about my thoughts on the purpose of Rally on my Agile Commons profile page or listen to my BlogTalkRadio interview.

Without a clear vision and mantra , it is hard to answer the question, Why do you develop and operate software?

As you and your software team/organization undoubtedly ask the core versus context questions in your business, I have been providing thinking tools for analyzing your portfolio in these turbulent times.  As Israel Gat pointed out in his recent post, “Can you afford the software you are developing?“

You do not need to take a long time to do this work, but it does provide a checklist of things to ask before you decide what product/code lines you might STOP DOING:

Things to ask before you decide what to stop doing:

1. What are your core values and vision?
2. What is your core purpose – Why? (this post)
3. What is your receipt for sustainable shareholder value?
   
4. What is your approach for getting your software organization leaner and getting stronger in 2009?

Once we have taken all the costs savings from distracting/mis-aligned efforts out of the portfolio, it is time to figure out how to increase the performance of your software development organization.  Jean and I will dedicate most of February to making the “Agile development cuts costs” case for groups that are working on core/mission critical efforts.

While you think about those arguments, I ask you to reflect on Bernard’s point:

Have you found your “Good Work?” and does that match up with your core work?

This last week I was with a number of our extra-large customers and prospects.  A common theme with these large and very successful high technology companies was the statement “growth hides all issues.” 

When we are growing rapidly, we all tend to focus on investment and hiring just to keep up and not limit the growth.

I remember this same feeling at BEA Systems in 2001 as the tech bubble popped. In our Boulder office, we found ourselves staring at 10 times more people than we started with in 1999.  Even though we took Extreme Programming (XP) teams into that growth, we ended up with a 100+ person and multi-location, waterfall process.  Our team reacted by putting back in XP engineering practices and large-scale, nightly, integrated builds to increase visibility and feedback.

Picture from Pegasus Communication - Leaders in Organizational Learning and System Thinking

A Two-Pronged Approach to Agile Adoption in a Recession

What is critically different about Agile adoption in this recession is the need for a two-pronged approach. In December, I wrote for Tech Target on two-pronged approaches to cutting budget while implementing Agile development. This is a systems thinking approach that is effective at breaking myopic thinking.

I was thrilled to see this approach used in the President’s stimulus package – short-term tax cuts to stimulate and provide relief coupled with future-looking government investments in more sustainable and equitable infrastructure. Without this type of approach, myopic thinking tends to increase the chances of “Fixes that Fail.”

It was a really good week last week.  As a result of this positive feedback, our team is going to devote a substantial amount of blog space during early 2009 to covering the strategy, approach, myths, and success stories around the use of Agile development to cut costs and simultaneously prepare for future growth.

Please stay tuned, link back your stories, comment, subscribe and consider your actions.

Further Reading:

• Agile Cuts Costs Through Productivity Improvements
• Where to Turn in a Down Economy?
• Sustainable and Successful – Top 10 Characteristics of an Agile Organization