Agile Development Blog

Michael Hugos, principal at Center for Systems Innovation [c4si], writes, speaks and consults on strategies for IT and business agility and mentors development teams. He spent six years as CIO of a multibillion dollar distribution cooperative developing a suite of supply chain and business systems that transformed the company’s operations and revenue model. He won the CIO 100 Award and Premier 100 Award for his work. He’s author of several books and writes an online column for CIO magazine called “Doing Business in Real Time.” We recently met with Michael at the Agile 2010 conference, which resulted in “Agile is Ready for the Enterprise” and sparked the idea for this blog post.

Rally asks: What issues and trends are you seeing across technology departments, development teams and in discussions with CIOs?

Michael Hugos answers: The example set by companies such as Google, Facebook and Netflix shows how companies can use iterative development to continuously enhance products and grow market share. This is being noticed by business and technology leaders in other companies and they are asking if they can do the same thing to drive development in their own companies. People realize that IT is right down the middle of everything a company does, and that traditional software release cycles of once a year, or even once a quarter, are not able to keep up with the pace of change and innovation these days.

Just like the word “athlete,” the word “Agile” grabs your attention; it sounds great. But moving from desire to reality always tests peoples’ commitment. A lot of companies are struggling with the all-too-common reaction of, “That’s not the way we do things here…” Agile approaches are interesting and fascinating to companies, but then there is the tendency to immediately criticize new ideas – we’re all prone to it. As soon as someone suggests a new way of doing something, we all think of 10 reasons why that can’t be done or why it won’t work.

Rally: What is driving enterprise adoption of Agile?

MH: To begin with, agility is no longer just a good idea; it’s now backed by law – the law of probability. This law says if a company can’t keep up with rapid rates of change in the world then its probability of success is getting smaller and smaller every day. And since companies need IT infrastructure and applications to operate, just as our bodies need a nervous system and muscles to move, IT agility is critical for a company to achieve business agility.

In the last few years, software tools have enabled executives to measure and track progress on Agile projects and to see the performance of Agile teams in widely dispersed geographical locations. That makes Agile methods more feasible for large companies. A pervading feeling exists throughout business that just about everything else has been tried and IT groups are still not really keeping up with the backlog of user requests. Users are starting to go around IT and do their own things using SaaS, social media and mashups to put together systems. So why not give Agile a try?

Rally: How do Agile methodologies help large organizations foster, regain or accelerate the pace of innovation?

MH: Agile practices offer the best way to improve communication and collaboration between business and IT. Meaningful innovation always starts with communication and collaboration. Another thing that Agile practices enable is the ability to try out new ideas and explore opportunities quickly without investing a lot of money up front. With more traditional approaches, companies invest a lot of time and money planning up front before they start something new. This is expensive. And since most new ideas don’t pan out in the end, this traditional approach makes it difficult (if not impossible) for companies to try out enough new ideas in a year to find that small handful of ideas that do work out and deliver the profits they are looking for.

I like to say that in this high-change and unpredictable economic environment, companies need to: “Think big, start small and deliver quickly.” That’s the best way to keep up, adapt and respond to change, and find the opportunities they are looking for. Agility means letting go of slow, deliberate decision-making in favor of quick, repeatedly-tested decisions. That’s why Agile methods are so appropriate for energizing companies and helping them develop innovative products and services.

Rally: How do you make a case for Agile and address the fears of risk-averse CIOs, CTOs or CEOs?

MH: First, I remind executives of something that has become a fact in the last 10 years: business operations and technology are so tightly intertwined that there is no meaningful distinction left between the two; you can’t do business without technology. That might seem obvious to many but, executives who have been around for a while (like me) may still remember the days when IT was just a back office operation.

Once people acknowledge this reality then I point out that, over the last 10 years, Agile practices have been thoroughly field tested and have an impressive track record for delivering success. There are software tools now, like Rally and others, that address Agile project management and reporting, business and IT collaboration, software testing, and the continuous integration of new software with existing systems infrastructure. So going Agile is not just a leap of faith anymore.

Agile is actually a better way to manage risk versus using traditional waterfall approaches. With Agile practices, big projects are divided into lots of smaller projects that build on each other. This enables people to employ short feedback loops, learn quickly and change plans in light of new information. Two of the biggest causes for failure in business and failure in new development projects is that companies have no inexpensive way to investigate new opportunities, and they blindly follow predefined project plans without change – even as the world itself keeps changing.

The IT profession is at a turning point: one group of IT practitioners has learned that agility is the way to go, but more traditional practitioners still call it radical. Yet, the traditionalists continue to apply the same old ways of doing things that result in the same old horrendously expensive, multi-year projects that produce systems barely better than what was there before, if they even work at all. More and more business executives are coming to the conclusion that the effective support of business agility is the main reason for their company to have an internal IT group. Otherwise, there are options now to just outsource IT operations to cloud computing vendors and get new applications from SaaS providers and social media.

Rally: What does the future hold for Agile and Lean development practices?

MH: Probably the biggest change will be analogous to what happens when a company grows and transitions from an entrepreneurial startup to an established business. When this transition happens, there is a need to become more pragmatic and less idealistic. In the Agile world, this means that “Scrum-but” will actually be the best way for most companies to adopt Agile methods. Each company will customize versions of Agile that best fit their needs and it will be some combination of practices from Scrum, XP, Lean, Kanban, etc. Even waterfall practices have some benefits which should be incorporated where they make sense. Agile practices will not be set in concrete; they will continue to evolve over time as companies learn more and the world keeps changing.

Another big change for Agile is the realization that Agile development is not an end in itself. The value of IT agility is its ability to drive business agility. In the end, agility is more about business than about IT. Instead of co-locating business people with development teams, we will embed IT people in business operating units and co-locate development teams with business people.

I talk about this in my most recent book Business Agility: Sustainable Prosperity in a Relentlessly Competitive World. Agile companies will become real-time organizations that use IT to drive a process of continuous focusing on and responding to opportunities and threats. They will employ IT to drive three continuous feedback loops that make their real-time operations possible. The first feedback loop (I use the Yin-Yang symbol), provides awareness of a changing environment and identifies threats and opportunities. The second loop (I use a sunflower because of how it constantly adjusts itself to follow the sun across the sky), provides balance and continuously adjusts existing operations and processes to fit changing circumstances. And the third loop (I use the leaping panther), provides agility in the sense that it is how companies create new processes and products to seize new opportunities. The figure below illustrates this.

This post will be split into two parts so that it, itself, will be half-assed with the suggestion of an additional half-ass to be delivered later :) And, in keeping with that spirit, the post mistakenly went live before it was ready for prime time. This time, I meant to push the ‘publish’ button…

Can you deploy Scrum to a test team?

Scrum is at its heart a product development process.  How can you leave the part of the organization – development – that actually makes product out of any Scrum deployment? Does it even make sense for other parts of the organization to be “doing Scrum” if development is somehow doing something else? Wouldn’t you be working towards what would be, at best, a half-assed deployment of Scrum?

Craig Larman and Bas Vodde in their wonderful book Practices for Scaling Lean & Agile Development certainly agree: “…a so-called test team Scrum is a contradiction in terms.” Ken Schwaber in The Enterprise and Scrum doesn’t seem to admit the possibility of deploying to functional groups – it’s projects he’s envisioning deploying to. For example, consider this advice for early goers of an enterprise-wide adoption of Scrum: “Establish preconditions that must be met before a project can use Scrum.”

The last 10 or so years of my career have been spent in big companies with very traditional structures: functional organizations with clear separation between development, test, usability, product management, etc; running projects that are very much plan-driven. Lots of agile practices that seem relatively straight-forward in other contexts aren’t in companies like this. Consider Schwaber’s notion of organizational deployment of Scrum again – this from the introduction of The Enterprise and Scrum: “This book is for those who want to use Scrum throughout their enterprise for product development.” It’s an awfully lucky convergence of thought and opportunity that would make such a deployment possible in large, traditionally organized companies. These sets of wholly distinct sub-organizations need to be both willing and able at essentially the same time. You might get a chance like that, but I wouldn’t hold your breath waiting for it.

You can start to see why that opportunity would be rare when you look at it from their perspective.  In taking a project-by-project focus in deploying Scrum to organizations like these – and assuming you’re holding firm to deploying every part of Scrum straight away – you’re essentially asking them to:

• Reconfigure their teams
• Change how they define and manage product scope
• Empower a single person to make scope decisions on each project
• Change how they plan their work
• Change how they approach their work in areas like development and testing
• And so on …

The point is that, even if limited to the context of a small set of pilot projects, they have to do all of this stuff first before they can realize the benefits of Scrum.

To me, this is exactly the same argument that we, as agilists, are very much accustomed to facing from development teams:  “We can do that feature but first we need to re-engineer the infrastructure to support it, which will take six months.”  We encourage teams making that argument to find ways to do just a bit of the refactoring to allow just a bit of the value to become realizable, rather than predicating all of the value on all of the refactoring.  Why can’t we make a similar argument in support of deploying Scrum to just a part of the organization?

What would Voltaire say?

One of my favorite lines – frequently quoted in optimization discussions but applicable equally well here – comes from Voltaire:  “Le mieux est l’ennemi du bien” (the best is the enemy of the good). “Best” is hard to define in any complex system like a large company but “good” seems a little more tractable. We should not let an inability to approach some notion of perfection prevent us from getting better.

A colleague was presented with this exact scenario a while back.  Representatives from a test group came to him asking if he would work with them to try Scrum.  He and I spent some time talking it over.

Things like product owner, product backlog and potentially shippable product increment looked like a tough fit for a test team, to be sure. But still, we thought of goals like “verify feature X” where the challenge to the team is to find a way to work together to get that done within the time-box of the Sprint. That might be a liberating shift in thinking after heavy doses of planning of the form: “We have a bazillion manual tests to execute. At 5 per hour, that’s bazillion/5 staff hours. With 20 FTE, that’s a bazillion/(20*5*40) weeks of testing.” Looking ahead to subsequent Sprints, we envisioned helping the test team bring some of their development partners into their Scrums – perhaps through broadening the notion of the verification goals to include “hardening” – finding and fixing bugs as a cross-functional team. And from there to the odd small feature, slowly working our way towards aligning the work of the test and development organizations in cross-functional Scrums. Even with such an odd scope of initial deployment, we could see potential benefits, including improved productivity through the iterate and reflect cycle, better planning and estimation and higher morale.  Not surprising, these are the same benefits we would suggest lay before any team looking to try Scrum.

Isn’t this the good that Voltaire would caution us against passing on?

Larman and Vodde have some great advice about how to go about ever more closely approaching the “potentially shippable product increment” goal of the Sprint through expanding the Definition of Done (DoD below):

“In general, these are the ways of expanding the DoD:

• Automate – for example, performance testing is automated
• Expand team cross-functionality – for example, a person with technical-writing skills joins the team”

That latter idea suggests a path to improvement that starts in development and then spreads over time to the remaining functions.  If we view Scrum deployment as being something we do in the context of projects and products, this is both natural and reasonable.  But, if we view deploying Scrum as something we do in the context of teams of people or if we view it simply as “transforming the world of work,” then why would we believe we have to start with any particular set of people?  Why not start with testing and grow our way towards development?

Would that be a half-assed approach to deploying Scrum?  Perhaps, but like Richard Dawkins’ half a wing or half an eye, maybe half an ass may prove a more useful incremental improvement than may be apparent at first glance.

So, can you deploy Scrum to a test team?

Sure, why not?

What do you think?

About the Author: Ed Willis has been a ScrumMaster, Product Owner, Scrum coach and trainer.  He is currently a developer in the telecommunications industry.

This week both Jean and I delivered talks on the Agile organization at Agile 2010 in Orlando. Whether you were able to attend one, both or neither, this post shares the handouts and materials that we used in the talks.

If you attended, please provide comments on what you liked, were puzzled by and might change in the future.

Jean’s work was a three-hour tutorial on learning models for managing the Agile organization.   She ran three exercises and provided a bibliography of books/resources that we have used here at Rally:

Jean in action at Agile 2010

• True North Example

• True North Exercise

• Strategy A3 Template

• ORID Exercise

• Personal and Corporate Visioning Exercise

• Sample Quarterly Agenda

• Bibliography

In addition to Jean’s talk, I presented an experience report on our use of Plan-Do-Check-Act (PDCA) at Rally.  This report tells a story of our evolution of strategy execution from Gazelles/Scrum to Lean/Agile.

We hope these resources provide you with ideas for scaling your own Agile efforts beyond their current levels.  Again, please comment on the blog with what you got from the materials or the talks.  We want to hear from you on this topic.

Ryan Martens is a tomato grower, founding board member of the Entrepreneurs Foundation of Colorado, and CTO at Rally Software Development.

Ah, Marketing.  Enter a world of countless project requests, numerous stakeholders, limited resources and rapidly changing market conditions.

Sound familiar?  In fact, marketers face a lot of the same challenges as development teams, and Agile can be a powerful way to alleviate those common issues and intelligently plan our work.

In steps 1-5, I’ll explain how Rally’s marketing team conducts our version of Release Planning.  In steps 6-10, I’ll explain how we run our iterations to meet those commitments. Our planning processes continue to evolve, though this method has worked for awhile now.

10 Steps to Successful Marketing using Agile and Lean Practices

1. We recognize that Marketing has challenges that are different from Development.

• There is no unique product owner.  For example, if we chose Sales, then we would always rank lead generation over branding, customer programs or analyst relations, and that could ultimately hurt our company. Therefore we have to use some best-guessing to prioritize our backlog and determine what is most important.
• We face hard event deadlines set far into the future.  Sometimes we have no choice but to commit to an event or sign a contract months ahead of time.
• Each team member has unique expertise, i.e. writing, event planning, PHP development and so forth.  So one shared backlog is inefficient.

Now that we’ve reviewed the challenges, we give ourselves permission to do what we need to do, have patience and adjust anything that isn’t working for us.

2. Conduct an ORID to learn from the past

Before planning for the next quarter, we hold a retrospective in the form of an ORID, “a means to analyze facts and feelings, to ask about implications and to make decisions intelligently”, a process created by the Institute of Cultural Affairs.  We gather as a team to share:

• Observations (O) – What do we know?
• Reflections (R) – How do we feel about this?
• Interpretations (I) – What does it mean for the organization?
• Decisions (D) – What are we going to do?

This strategic overview helps set context for us to prioritize our focus for next quarter.

3. Align ORID decisions with company strategy

At Rally, we conduct quarterly and annual planning using the Plan Do Check Adjust methodology as explained in Getting the Right Things Done.   As we look at the overall company direction and goals, we keep these in mind as we hold planning at our own level.   Ideally, our major commitments support and align with company strategy. This also helps inform our “stop doing” list.

4. Poll our stakeholders

As part of determining quarterly commitments, we poll our major stakeholders for their top requests.  We use a Google survey to rank these requests by importance, size each request and bring these epics into our release planning meeting, to be included as part of our ranked backlog.

5. Conduct Release Planning to prioritize and agree on quarterly commitments

Now that we have all of our inputs, we hold our quarterly Release Planning session.  We write each epic on a sticky note and look at all of the possible work we could do this quarter.  Then, we evaluate epics based on importance taking company goals, stakeholder wishes, market realities like conferences and our own passions into consideration. We decide what we can realistically commit to, and agree as a team.  We keep in mind that making and meeting commitments is a huge deal, and we try really hard not to over-commit.

Part 2 – Meet our Marketing Commitments

6. Create a task board

Since our marketing team is mostly co-located, we pin up several large sheets of paper to use as a task board.  This is where we review our commitments on a daily basis as a sanity check that our stories are prioritized correctly and that we are tackling the right work as the quarter progresses.

As a team, we write our quarterly commitments on the task board with the definition of done assigned to each one.  We also include our “foundational” work – recurring work like website updates, online ad campaigns, field events, press releases and other important work that we need time to do.

Note: this is not a Kanban board because we do not have a shared backlog nor do we have a team-wide WIP limit.  As we break into smaller project teams that do share a backlog, we often use AgileZen to manage this work.

7. Hold iteration planning every 2 weeks

Every 2 weeks, we hold an Iteration Planning meeting.  Each team member has her own sticky note color, creates stories on those notes and manages her own prioritized backlog using T-shirt sizing to roughly estimate each story.  In this hour-long meeting, we begin by expressing appreciation for team members who gave exceptional assistance.  Then we hold a brief retrospective on what worked and what should change for the next iteration.  Finally, we each read out our prioritized stories for the iteration, putting them on the task board’s backlog.  This gives everyone visibility to what’s happening, identifies if someone is over-committed and lets the team swarm any epics with looming deadlines.

8. Conduct a daily stand-up

At the same time each day, we hold a stand-up meeting (with a consistent conference call #) that is at most 10-15 minutes long.  We form a semi-circle in front of our task board and share no more than 2 cross-cutting significant actions or take-aways from the day before, no more than 2 that we are planning to accomplish that day, and whether our work is blocked by any issues beyond our control.   As we start working on stories throughout the iteration, we move them from the backlog into their section of the task board to show what we are working on.  When the story is complete, then we move it to a place on the task board labeled “Done”.  Once the commitment’s Definition of Done is met, we check off that commitment and feel good about completing it.

9. Be patient as things change

It would be lovely if nothing changed during the iteration, but that just doesn’t happen.  The goal is ultimately to respond to change rather than cling to an outdated plan.  As new opportunities arise, new time-sensitive information appears and new requests are made, so our iteration work changes and that’s ok.  We try to just document what we’re working on and create new stories so that we can make intelligent prioritization decisions during the course of the iteration.

10. Retrospect, inspect and adapt

As we keep running our iterations and fulfilling our commitments, we are always looking for ways to improve them.   Ultimately, we’re using Agile to improve the quality of our work life by using objective, smart ways of planning how we spend our time. And we’re learning a lot from the journey.

Dear Readers,

Writing or receiving a break-up letter can be fairly daunting or shattering, depending on which end of the letter your name appears. That letter puts a pretty hard stop to a relationship. It’s communicating detachment and finality. It can create a lot of pain whether intended or not. In contrast, a love letter is uplifting. The endorphins fly! Someone is revealing their attraction for you, and their hopes and wishes for a future with you.

Now, there is a reason I have these letters on my mind. I’ve just returned from Rally’s Agile Leadership Forum – a great gathering of people eager to lead successful Agile transitions in their organizations. The event included a lively presentation from Forrester Research’s Senior Analyst Dave West: “Agile Adoption – Research Findings on the Adoption of Agile.” (You can find some of Dave’s data in the “Forrester Wave: Agile Development Management Tools, Q2 2010″). We also enjoyed an inspirational talk from our CTO Ryan Martens, called “Moving Agile Beyond Software.” These great presentations were followed by breakout sessions and a panel discussion about Agile challenges. Now, how to end the event?

As emcee of the forum, I not only kicked off the event, but it was my job to bring closure to the gathering as well. How can we have people walk away with thoughts about Agile? Why are they interested in the first place, and where do their concerns lie?  I was inspired by a video I recently saw about “breakup letters.” The Breakup Letter is a design research tool that Smart Design uses to understand the emotional connection between people and their products, services, and experiences. One person broke up with his cell phone, another, her single-cup coffee maker.

Now, just how does this relate to the Agile Leadership Forum? I liked the concept of the breakup letter, but I decided to entirely flip the idea and close the event by asking everyone to write love letters instead. In the spirit of Cyrano de Bergerac, I asked each table of participants to work together in crafting a “Dear Agile” letter. In this letter, they were to convey their attraction to Agile. And, they were to reveal where they were concerned about as well. All letters were to be from a secret admirer :-)

Once the groups began to read their letters, I knew we were on to something. Though I don’t have the reading of the letters on video, here are a few examples of our “Dear Agile” love letters.

Run this exercise in your own group to find out what the Agile “lure” looks like and also what the “turn-offs” might be.

Breathlessly awaiting your comments,

Jean

p.s. If you want to read some of the transcribed texts of the love letters, read on!

__________________________

Dear Agile,

I have admired you from a distance for some time. Waterfall and I are in the process of an ugly breakup. There is so much about you I need to know. My friend says great things about you. You are so simple and straightforward– no mind games like Waterfall.

This won’t be simple. Waterfall still has clothes at my place. My Facebook status is confused.

In the relationship as we get to know one another, we will have to know each other carefully– co-locating right away? Are we sprinting too fast?

Be gentle with me.

Looking forward to a rapidly developing future.

xoxoxo,

Secret Admirer

__________________________

Dear Agile,

I love you because you offer quick cycles, better quality, and better teamwork. From the first time I saw you, I thought I could begin saving money and add business value.

But, fair Agile, you are not so simple. I’ve heard you are a micro-manager. I don’t totally understand you. Some people are confused by you. On the surface, you sound so perfect and simple, but the more I get to know you the more questions I have.

But, among all my choices, I choose you. You promote collaboration, and allow me to turn things around quickly. You’ve helped me trim weight and stay lean. Don’t disappointment me, I trust you!

With all my love,

Megedá

___________________________

Dear Agile,

I loved you from the first moment I saw you, I loved your fast, speedy releases and that you don’t come with a lot of baggage or documentation. You’re simple and down to earth. You are a great communicator. I always know where you are and my friends love you, too.

I am, however, a bit concerned that not everyone accepts our relationship. I am worried that as my job continually grows and my needs scale up, whether you can handle the increasing challenges. And I’m concerned whether I can afford you… Our relationship and your attachments are what intrigue me the most.

Looking forward to spending more time with you and getting to know you better.  – Your secret admirer.

___________________________

Dear Agile,

We love you, we think you are awesome – for the following (bulleted) reasons:

• Agile accepts changes and encourages frequent changes
• Agile can start implementation before full requirements are known.

We do however have a few problems with you agile –

• Handling cultural change in the organization
• Does not solve all our issues
• Makes distributed teams harder to work with

- Your secret admirer -

Matt Phillips is a Senior Project Manager who has spent the last few years helping shape the Agile development process at Lulu.com. He currently heads up the Lulu Project Management Office and has spent several months setting up Agile practices in Lulu’s India office, based in Bangalore. In advance of his executive panel discussion at Rally’s Agile Success Tour in Raleigh, NC, we sat down with Matt to ask him 5 questions about Agile.

1. How have you implemented Agile across your organization?

We’ve rolled Agile out among all of our distributed teams, which are located in Raleigh, NC, the Ukraine and India. The time zones have historically been a challenge, so we had our remote teams spend several weeks in the Raleigh office working through daily Scrums. Now, they’re essentially as included in the process as possible. We use video conferencing for daily Scrums and to schedule iteration planning. All the teams collaborate to define stories, determine velocity, and plan iterations. We use Rally to make projections, track our velocity, and get visibility into the health of our projects. The metrics have become indispensible for judging how we’re doing, making accurate projections, and delivering upon our commitments.

2. What was your #1 reason for adopting Agile development?

Lulu adopted Agile at a point where the company was very much in start-up mode. The ideas were coming at a frenetic pace and the engineering team size was poised to expand. Agile methodologies were a good fit for Lulu’s culture and environment. The concepts of short iterations and regular release cycles paired with Scrum provided a quick time-to-market period for new ideas. At the same time, by adopting Agile methodologies, Lulu gained increased insight into the development team’s progress and performance as the team grew and feature sets became more complex.

3. What has been the biggest benefit of adopting Agile?

The metrics and amazing visibility we have into development projects. This is especially important for a team that’s 9,000 miles away. We have visibility into how they’re progressing on features, what’s coming next in the roadmap, and really flushing out what the product backlog looks like and where we’re headed.  Prior to implementing Agile, it was very hard to sync-up  (because of the 9 ½ hour time difference), maintain a feedback loop and foster collaboration with teams so far away.

4. What one piece of advice would you give to new Agile teams?

My advice would be to ease into it – kind of like steering a cruise ship, not turning on a dime. Start with familiar concepts and gradually introduce Agile practices over time. We started with familiar ideas like release dates, associated task lists, estimations, and tracking criteria. Then, we used a phased approach to introduce the concepts of iterations, story points and relative sizing, velocity, and ranking. We continue to work toward more granular inputs to smoothly coordinate roles, tools and dependencies within Rally as we go along to continuously perform at higher levels and get better outputs.

5. How can you tell that Agile is successful at Lulu.com?

One of top ways I can tell that Agile is successful in our organization is that people, even outside of engineering, are speaking in story points. That tells me that Agile has really taken hold. Using story points and velocity for our release planning makes it easy to arrive at a date that everyone is comfortable with. On top of that, our track record since adopting Agile shows that we’ve been delivering on our commitments every time.

Managing the right goals in software development can have a major impact on your team’s success. You get what you measure. In this regard, the goals you choose can be a defining factor in whether your organization’s Agile adoption thrives or dives. So, what are your choices?

A story about goal-setting

Imagine you are a runner. You have a running coach defining targets for you. Your coach chooses miles and seconds in order to measure you. These metrics turn out to be useful; they are universally understood by you, your coach and your competitors. That’s a good thing. A target goal exchange around these metrics might look like the following:

Coach: “I need you to run a 2-minute mile.”

You: “Uh, I’m not sure I can do that. In fact I don’t think it has ever been done before by any land animal other than a wildebeest, a pronghorn antelope or a cheetah. And, I’m not a wildebeest.”

Coach: “You’re not listening. I already told Sports Illustrated you will. Don’t make me look bad.”

You: “Seriously I’m pretty sure that’s just not humanly possible. I’m no Usain Bolt and I’m not sure even he could sustain a 2-minute mile.”

Coach: “Hey, I set the goals. Run that mile in 2 minutes. Just hunker down and work harder. 2-minutes is the target goal. If you don’t meet it, I’m kicking you off the team.”

You: (muttering under your breath to self, “I can’t do it, you aren’t listening to me, and you are a moron. Why do I even bother to open my mouth?”)

Why goals matter

Get the idea? I’ve been reading John Seddon’s Freedom from Command and Control: Rethinking Management for Lean Service. My colleague Alan Atlas recommended the book. Then my manager Ryan Martens became very interested which led to a book appearing on my desk. Soon after, Karl Scotland mentioned it in an informal conversation about Lean and Kanban. Time to pop it to the top of my reading stack!

Seddon sets a context about Purpose driving Measures that then drive Methods. The combination of purpose and measures start to look like goals. And Seddon breaks goals into two types: Target Goals and Capability Goals. While John’s book is not specifically about software, I could see how we use and abuse goals in our software world. So I decided to delve deeper.

Target goals are arbitrary and doom-laden

Target goals are set to measure hopeful results, often an arbitrary number. Think about our runner story and the 2-minute mile. That target goal was  set by someone else. A target goal typically does not account for your strengths, capabilities , availability and skills. Target goals don’t serve you. They don’t serve organizations either. And yet we cling to them. Consider if, as a developer, I hear the following from my manager:

“Jean! You missed the release deadline we gave you. We told marketing and sales who then told our customers that dev would deliver this specific functionality on this specific date. You’re ruining everything. We’re doomed I tell you, doomed! And all because YOU didn’t hit YOUR deadline.”

Uh, it wasn’t my deadline. And yet, missing that goal is now rippling negatively through the organization. This is a lose-lose situation all due to expectations hinged to an arbitrary target goal.

So why do target goals exist?

In many hierarchical, command-and-control software organizations, target goals are how deadlines are set. As Seddon points out, use of ineffective target goals usually arises due to an impossible purpose coupled with non-value add methods. The measures (or goals) are meant to drive teams to the impossible purpose. A command-and-control organization believes that goals must be set top down. The negative impact of these goals is inextricably woven into unrealistic expectations, death marches, and non-value add methods for controlling the work toward the goals.

Enter the capability goal

Let’s replay the runner scenario again.

Coach: “There is a race coming up, a half marathon. That means we need to start training now. What is your time currently?”

You: “I haven’t been really running lately. Rather than guess my times, I’ll start running now. Do you want to know in terms of how long it takes me to run a mile, or how far I can run in a specific timebox?”

Coach: “Let’s start with how you finish a mile. What is your typical way of getting going?”

You: “First, I do a walk/run combination. Then I can give you my actual mile capability in a week.”

Coach: “Okay, let’s set up a plan and together set some sort of goals based on your capabilities. That will help me guide you and figure out when we are faltering. I know you can do this. I’m going to work with you starting right now.”

You: (muttering under your breath, “This is amazing; yea me! I am going to finally put my passion for running to work, keep improving, and run my first half marathon. I love my coach!)

Capability goals are based on real data

Together, the runner and the coach watch the capabilities of the runner to define goals. The purpose of the goal is clear. Measures are set up that usefully help determine the runner’s progress. The method (or running regimen) for attaining the goal is then applied. Simple.

We use this same approach in Agile teams. A team lead or project manager does not set an arbitrary target goal. Rather, the Product Owner sets an overall purpose/vision. The Agile lead, often referred to as the ScrumMaster, works with the team to assess what goals can be reached based on a team’s capabilities and availability. From iteration to iteration, the team provides feedback on what it is able to complete based on its capabilities. The Product Owner absorbs this information and sets expectations outside the team. The process of continuous improvement continues. The team declares, “Based on what we tend to complete, and given the purpose from the Product Owner, we are going to improve our methods in the following ways.”

When good capability goals go bad

There’s a Gary Larsen “Far Side” cartoon of an open refrigerator with a bowl of glop inside holding a gun. The caption? “When good potato salad goes bad”.

Unfortunately, I’ve seen good capability goals go bad in a number of organizations. Listen to this conversation:

Program Director: “I see that our Agile teams are all now using these things called story points to track their work.”

Me: “Yes, it’s working well. Each team has a good sense of what it is capable of delivering iteration over iteration. That helps me figure out how to help remove impediments, where bottlenecks are, and where any team may need us to better resource them. It’s great!”

Program Director: “Hmmmm.”

Me: (muttering under my breath: “That ‘Hmmmm’ just doesn’t sound good. I think I’m about to get sick.”)

Program Director: “So, what’s the highest story point commitment a team has made?”

Me: “27.”

Program Director: “Great!  I’m setting 27 points as the target goal for all teams to commit to for each iteration. Every point will represent 3 hours of work. This precision will help me calculate exactly what we’ll deliver when. Plus, I’ll know which teams are really working and which ones are slackers.”

Me: “Hmmmm.”

Program Manager: “So, let’s get out there and commit! This Agile stuff is great!”

Me: (muttering under my breath, “We’re doomed, I tell you doomed!”)

What just happened must not be allowed

As you roll out your organizational Agile adoption, you’ll encounter many challenges. Pay attention to the usefulness of capability goals. Insist on using them.  Non-Agile organizations won’t like this. At times, you may feel like Sisyphus pushing a rock up hill for all eternity. Be strong. Don’t allow Agile measurement abuse. Create team and organizational trust.  Remove target goals and embrace the incredible value of capability goals. And, don’t allow any backslide. Don’t let your capability goals devolve slowly into target goals in sheep’s clothing.

You: (muttering under your breath, “You know? This MAY just work. Capability goals versus target goals. For the sake of my team, our organization, and the value we deliver to our customers, I’m going to do it!”)

Brad Murphy, CEO of Gear Stream, gave a keynote presentation at last week’s Agile Success Tour. Brad’s vision includes helping organizations transition from project-driven governance to Lean, continuous flow, product line execution. We sat down with Brad to ask him 5 questions about Agile.

1. What’s the most valuable thing you’ve learned by helping organizations adopt Agile practices?

The dynamics surrounding the adoption of Agile practices are changing in ways that are very different from even two or three years ago. In parallel with implementing Agile practices and establishing effective Agile teams, we’re helping organizations connect Agile philosophies and values to vital stakeholder groups. We’ve recognized that actively involving all stakeholders early in the process is key to ensuring that core Agile teams are able to achieve expected results. Gaining the support of all collaborators in the organization dramatically helps core Agile teams successfully transform and deliver outcomes. Involving the following stakeholders is vital, including:

• The core Agile team
• Tooling, Infrastructure and Release Management – the architecture side of the equation.
• Governance and PMO groups – more than half the time, this group builds the roles of the project managers, Agile teams and the Scrum Master. They consider how Agile teams work off backlogs. There’s huge potential for misalignment here.
• Product Management – since this is frequently a group that doesn’t exist, the responsibilities here aren’t well defined. Loosely, I’ll describe this as a group of individuals responsible for defining business value. This group has to be aligned with the way Agile works.
• Executive Management – all of the focus on Agile change with the groups listed above must be explicitly aligned with the strategy and values articulated by senior executives. Too often, Agile is pursued based on “generic” benefits like speed, responsiveness, quality… while these are “good,” they’re not explicit enough.  We must challenge executives to express exactly how they will measure/recognize their own corporate goals and then map these back to the things we choose to emphasize in an Agile transformation.

Responsiveness would be #1. Having the ability to course-correct and change directions quickly in light of new mandates and directions. A close #2 is innovation.  A distant #3 is productivity.

Gear Stream’s clients are realizing compelling organizational realignment by refocusing the entire value chain on external customers, as opposed to the often dysfunctional inward focus of many organizations.  By aligning the entire value chain on external customers, our clients are able to more predictably influence customer satisfaction and marketshare objectives.

I’ll give the advice I’m most passionate about. Agile teams need to challenge the rest of the organization to rethink and reconsider how work happens up and down-stream to them in order to fully exploit what they’re capable of achieving. It’s really about how Agile teams can actively promote, influence and change an organization beyond just what they do as a team. They really can – and do – influence and engage the rest of the organization. The failure pattern of Agile is almost always the same: we’ve built an Agile team, but failed to build the adjacent stakeholder teams in the process – then, the Agile team gets frustrated and the organization loses momentum and enthusiasm for Agile. Agile teams must have the ability to influence stakeholder teams, and ultimately, the baseline infrastructure of the entire organization.

5. How do you define Agile success? (Can you do it in 113 characters?)

Building, delivering & sustaining outcomes that customers r continually thrilled by, driving profit & value 4 all

Yesterday, the Agile Journal posted their May edition of the online publication with the above focus.  To support their release, they asked us to share our opinion on the topic.  Here is our response:

Ryan Martens: Age of Reconsideration, Reform and Regeneration

The last decade marshaled in a new empirical way of working with increasingly complex, interconnected and highly-critical software-based systems – Agile.  We are entering a period of reconsideration, reform, and regeneration in software, systems engineering and project management.  Agile is working, Lean Software and System is working, and the combination is starting to prove very powerful with regard to throughput and workers.  The benefits of autonomous work, engagement and mastery are driving systemic improvements in our way of working and growing to meet complex challenges of our world.

These results illuminate a future vision that has the potential to expand our current notions of Lean and Agile from software teams and into real organizational agility.  As a result, there is a chance to unite and unify many communities under the guiding ideas of flow, pull, and value.  All of these communities are being drawn in and starting to play well.  These are beautiful days with all the implications to CMM/SEI, Agile, Scrum, Lean/LEI, and PMI/PMBok communities yet to be determined.

In the first half of this decade, look for collaborating across boundaries, seeing larger systems and groups working hard to create their future realities.  Following that period, look for messy consolidation as the winners of this new platform emerge for a new golden age of networked, software product and system development. Together we’ll be focused on the problem domain of global scale difficulties in governance, cyber-warfare, energy, water, communication, commerce, medicine, climate, transportation and nano-technology.

Ryan Martens is an amateur triathlete,  founding board member of the Entrepreneurs Foundation of Colorado, and CTO at Rally Software Development.

Jean Tabaka: Let the System Talk

Thinking about our path with Lean, I’m compelled to draw upon research I’ve been doing in Systems Thinking and, more recently, what I’ve been learning in Systems Engineering.

In Systems Thinking, we recognize a world of system archetypes based on the dance of balancing loops, reinforcing loops and the outside agents that may cause them to transition. Lean, as a system of thinking, has certainly responded to systems that rely too much on take-make-waste. A set of negative reinforcing loops: the more you waste the less you have to take and make. Outside agents, the scarcity versus abundance of materials, has led us to Lean. Lean principles and practices create a positive system wherein the more we reduce waste the more value we get which in turn reinforces more waste reduction. It is a reinforcing loop propelled by continuous improvement.

Recently, I attended the Lean Software and Systems Consortium’s 2010 conference in Atlanta. What a revelation. From James Sutton’s talk on Lean Systems Engineering, I added new vocabulary that I think will become critical to Lean’s future.

Will Lean be our best source of practices and principles in the future? That depends on what will be guiding our systems:

• Scarcity
• Abundance
• Desperation
• Conformity

Once we have clarity about what guides our system, we can understand more about the system in which we are operating:

• Simple
• Complicated
• Complex
• Chaotic

Lean has steadfastly addressed pressures of scarcity and hence a system of complexity. That brings me to Dave Snowden’s work captured in Cynefin, a Welsh word he has used to describe a framework of problems, situations and behaviors in these four systems. For our world of complex systems, Lean provides the perfect high-level thinking for what we must embrace: emergent practices informed by, as Snowden puts it, “sense-making”

As we move into the next 10 years of Lean, I fervently believe that our sense-making must inform us about what supports emergence that responds to complexity. The practices will follow. For now, let us concentrate on the systems in which we operate, what outside agents or pressures are guiding our systems, and how we can best continue to formulate and hold dear the practices that will naturally emerge.

Jean Tabaka is a wine enthusiast, author and Agile Fellow at Rally Software Development.

Scott Killen is an Agile Practice Leader at PayPal and the President of Agile Austin. He has extensive experience managing all aspects of the software development process, from a twinkle-in-the-eye to final release. In advance of his participation as a panelist at our upcoming Dallas Agile Success Tour, we sat down with Scott to chat about Agile.

1. How have you implemented Agile across your organization?

We have a three-year vision for PayPal to implement Agile development across our enterprise. In year one, we are focusing on basic practices. The idea is to lay a solid foundation for building mature teams. In year two, we’ll focus on making teams successful. That is, using the practices in a reinforcing way to build teams that predictably deliver acceptable products. In year three, we want to focus on optimization. That is, aggressively practicing incremental and continuous improvement of our Agile practices.

To baseline practices and measure improvement in year one, we have a metric that we call “Team Maturity.” This is a weighted score that reflects the number of fundamental practices a team employs and their longevity in terms of iterations completed.

We are in the process of defining the metrics we wish to use to define a successful team. It is important to us that these metrics be methodology neutral. Although not formalized, I expect that some measure of time-to-market and released-defect counts will be used.

2. What was your #1 reason for adopting Agile development?

We want more flexibility in our development process.  We seek flexibility to make late-cycle changes to develop the best product and to decrease time-to-market for those products.

3. What has been the biggest benefit of adopting Agile?

We’re looking forward to receiving the big payoffs. But, surveys generally indicate that we are more effective when we develop in Agile projects, our quality is higher, and our developers are happier.  Cross-functional communication is cited as the big win in these surveys.

4. What one piece of advice would you give to new Agile teams?

Be sure to properly set management expectations. You will not get productivity or quality gains at first, and, in fact, your productivity will decline for about six iterations until everyone begins to learn how to work collaboratively within the Agile framework. Agile development does not come without pain. I’m reminded of the expression “Pain is weakness leaving the body.”  That’s Agile – painful, but the results are worth it.

5. How do you define Agile success? Can you do it in 113 characters?

Happier customers measured by Net Promoter scores. Happier developers measured by lower turnover.

Follow this thread on Twitter - We asked Scott to define Agile success in 113 characters or less as part of a challenge we’re hosting on Twitter. What is your definition of Agile success in 113 characters or less? Tweet your answer to @RallySoftware using #AgileSuccess. The top five tweets received by May 17th will be featured in a follow-up blog post.