Agile Development Blog

Michael Hugos, principal at Center for Systems Innovation [c4si], writes, speaks and consults on strategies for IT and business agility and mentors development teams. He spent six years as CIO of a multibillion dollar distribution cooperative developing a suite of supply chain and business systems that transformed the company’s operations and revenue model. He won the CIO 100 Award and Premier 100 Award for his work. He’s author of several books and writes an online column for CIO magazine called “Doing Business in Real Time.” We recently met with Michael at the Agile 2010 conference, which resulted in “Agile is Ready for the Enterprise” and sparked the idea for this blog post.

Rally asks: What issues and trends are you seeing across technology departments, development teams and in discussions with CIOs?

Michael Hugos answers: The example set by companies such as Google, Facebook and Netflix shows how companies can use iterative development to continuously enhance products and grow market share. This is being noticed by business and technology leaders in other companies and they are asking if they can do the same thing to drive development in their own companies. People realize that IT is right down the middle of everything a company does, and that traditional software release cycles of once a year, or even once a quarter, are not able to keep up with the pace of change and innovation these days.

Just like the word “athlete,” the word “Agile” grabs your attention; it sounds great. But moving from desire to reality always tests peoples’ commitment. A lot of companies are struggling with the all-too-common reaction of, “That’s not the way we do things here…” Agile approaches are interesting and fascinating to companies, but then there is the tendency to immediately criticize new ideas – we’re all prone to it. As soon as someone suggests a new way of doing something, we all think of 10 reasons why that can’t be done or why it won’t work.

Rally: What is driving enterprise adoption of Agile?

MH: To begin with, agility is no longer just a good idea; it’s now backed by law – the law of probability. This law says if a company can’t keep up with rapid rates of change in the world then its probability of success is getting smaller and smaller every day. And since companies need IT infrastructure and applications to operate, just as our bodies need a nervous system and muscles to move, IT agility is critical for a company to achieve business agility.

In the last few years, software tools have enabled executives to measure and track progress on Agile projects and to see the performance of Agile teams in widely dispersed geographical locations. That makes Agile methods more feasible for large companies. A pervading feeling exists throughout business that just about everything else has been tried and IT groups are still not really keeping up with the backlog of user requests. Users are starting to go around IT and do their own things using SaaS, social media and mashups to put together systems. So why not give Agile a try?

Rally: How do Agile methodologies help large organizations foster, regain or accelerate the pace of innovation?

MH: Agile practices offer the best way to improve communication and collaboration between business and IT. Meaningful innovation always starts with communication and collaboration. Another thing that Agile practices enable is the ability to try out new ideas and explore opportunities quickly without investing a lot of money up front. With more traditional approaches, companies invest a lot of time and money planning up front before they start something new. This is expensive. And since most new ideas don’t pan out in the end, this traditional approach makes it difficult (if not impossible) for companies to try out enough new ideas in a year to find that small handful of ideas that do work out and deliver the profits they are looking for.

I like to say that in this high-change and unpredictable economic environment, companies need to: “Think big, start small and deliver quickly.” That’s the best way to keep up, adapt and respond to change, and find the opportunities they are looking for. Agility means letting go of slow, deliberate decision-making in favor of quick, repeatedly-tested decisions. That’s why Agile methods are so appropriate for energizing companies and helping them develop innovative products and services.

Rally: How do you make a case for Agile and address the fears of risk-averse CIOs, CTOs or CEOs?

MH: First, I remind executives of something that has become a fact in the last 10 years: business operations and technology are so tightly intertwined that there is no meaningful distinction left between the two; you can’t do business without technology. That might seem obvious to many but, executives who have been around for a while (like me) may still remember the days when IT was just a back office operation.

Once people acknowledge this reality then I point out that, over the last 10 years, Agile practices have been thoroughly field tested and have an impressive track record for delivering success. There are software tools now, like Rally and others, that address Agile project management and reporting, business and IT collaboration, software testing, and the continuous integration of new software with existing systems infrastructure. So going Agile is not just a leap of faith anymore.

Agile is actually a better way to manage risk versus using traditional waterfall approaches. With Agile practices, big projects are divided into lots of smaller projects that build on each other. This enables people to employ short feedback loops, learn quickly and change plans in light of new information. Two of the biggest causes for failure in business and failure in new development projects is that companies have no inexpensive way to investigate new opportunities, and they blindly follow predefined project plans without change – even as the world itself keeps changing.

The IT profession is at a turning point: one group of IT practitioners has learned that agility is the way to go, but more traditional practitioners still call it radical. Yet, the traditionalists continue to apply the same old ways of doing things that result in the same old horrendously expensive, multi-year projects that produce systems barely better than what was there before, if they even work at all. More and more business executives are coming to the conclusion that the effective support of business agility is the main reason for their company to have an internal IT group. Otherwise, there are options now to just outsource IT operations to cloud computing vendors and get new applications from SaaS providers and social media.

Rally: What does the future hold for Agile and Lean development practices?

MH: Probably the biggest change will be analogous to what happens when a company grows and transitions from an entrepreneurial startup to an established business. When this transition happens, there is a need to become more pragmatic and less idealistic. In the Agile world, this means that “Scrum-but” will actually be the best way for most companies to adopt Agile methods. Each company will customize versions of Agile that best fit their needs and it will be some combination of practices from Scrum, XP, Lean, Kanban, etc. Even waterfall practices have some benefits which should be incorporated where they make sense. Agile practices will not be set in concrete; they will continue to evolve over time as companies learn more and the world keeps changing.

Another big change for Agile is the realization that Agile development is not an end in itself. The value of IT agility is its ability to drive business agility. In the end, agility is more about business than about IT. Instead of co-locating business people with development teams, we will embed IT people in business operating units and co-locate development teams with business people.

I talk about this in my most recent book Business Agility: Sustainable Prosperity in a Relentlessly Competitive World. Agile companies will become real-time organizations that use IT to drive a process of continuous focusing on and responding to opportunities and threats. They will employ IT to drive three continuous feedback loops that make their real-time operations possible. The first feedback loop (I use the Yin-Yang symbol), provides awareness of a changing environment and identifies threats and opportunities. The second loop (I use a sunflower because of how it constantly adjusts itself to follow the sun across the sky), provides balance and continuously adjusts existing operations and processes to fit changing circumstances. And the third loop (I use the leaping panther), provides agility in the sense that it is how companies create new processes and products to seize new opportunities. The figure below illustrates this.

This post will be split into two parts so that it, itself, will be half-assed with the suggestion of an additional half-ass to be delivered later :) And, in keeping with that spirit, the post mistakenly went live before it was ready for prime time. This time, I meant to push the ‘publish’ button…

Can you deploy Scrum to a test team?

Scrum is at its heart a product development process.  How can you leave the part of the organization – development – that actually makes product out of any Scrum deployment? Does it even make sense for other parts of the organization to be “doing Scrum” if development is somehow doing something else? Wouldn’t you be working towards what would be, at best, a half-assed deployment of Scrum?

Craig Larman and Bas Vodde in their wonderful book Practices for Scaling Lean & Agile Development certainly agree: “…a so-called test team Scrum is a contradiction in terms.” Ken Schwaber in The Enterprise and Scrum doesn’t seem to admit the possibility of deploying to functional groups – it’s projects he’s envisioning deploying to. For example, consider this advice for early goers of an enterprise-wide adoption of Scrum: “Establish preconditions that must be met before a project can use Scrum.”

The last 10 or so years of my career have been spent in big companies with very traditional structures: functional organizations with clear separation between development, test, usability, product management, etc; running projects that are very much plan-driven. Lots of agile practices that seem relatively straight-forward in other contexts aren’t in companies like this. Consider Schwaber’s notion of organizational deployment of Scrum again – this from the introduction of The Enterprise and Scrum: “This book is for those who want to use Scrum throughout their enterprise for product development.” It’s an awfully lucky convergence of thought and opportunity that would make such a deployment possible in large, traditionally organized companies. These sets of wholly distinct sub-organizations need to be both willing and able at essentially the same time. You might get a chance like that, but I wouldn’t hold your breath waiting for it.

You can start to see why that opportunity would be rare when you look at it from their perspective.  In taking a project-by-project focus in deploying Scrum to organizations like these – and assuming you’re holding firm to deploying every part of Scrum straight away – you’re essentially asking them to:

• Reconfigure their teams
• Change how they define and manage product scope
• Empower a single person to make scope decisions on each project
• Change how they plan their work
• Change how they approach their work in areas like development and testing
• And so on …

The point is that, even if limited to the context of a small set of pilot projects, they have to do all of this stuff first before they can realize the benefits of Scrum.

To me, this is exactly the same argument that we, as agilists, are very much accustomed to facing from development teams:  “We can do that feature but first we need to re-engineer the infrastructure to support it, which will take six months.”  We encourage teams making that argument to find ways to do just a bit of the refactoring to allow just a bit of the value to become realizable, rather than predicating all of the value on all of the refactoring.  Why can’t we make a similar argument in support of deploying Scrum to just a part of the organization?

What would Voltaire say?

One of my favorite lines – frequently quoted in optimization discussions but applicable equally well here – comes from Voltaire:  “Le mieux est l’ennemi du bien” (the best is the enemy of the good). “Best” is hard to define in any complex system like a large company but “good” seems a little more tractable. We should not let an inability to approach some notion of perfection prevent us from getting better.

A colleague was presented with this exact scenario a while back.  Representatives from a test group came to him asking if he would work with them to try Scrum.  He and I spent some time talking it over.

Things like product owner, product backlog and potentially shippable product increment looked like a tough fit for a test team, to be sure. But still, we thought of goals like “verify feature X” where the challenge to the team is to find a way to work together to get that done within the time-box of the Sprint. That might be a liberating shift in thinking after heavy doses of planning of the form: “We have a bazillion manual tests to execute. At 5 per hour, that’s bazillion/5 staff hours. With 20 FTE, that’s a bazillion/(20*5*40) weeks of testing.” Looking ahead to subsequent Sprints, we envisioned helping the test team bring some of their development partners into their Scrums – perhaps through broadening the notion of the verification goals to include “hardening” – finding and fixing bugs as a cross-functional team. And from there to the odd small feature, slowly working our way towards aligning the work of the test and development organizations in cross-functional Scrums. Even with such an odd scope of initial deployment, we could see potential benefits, including improved productivity through the iterate and reflect cycle, better planning and estimation and higher morale.  Not surprising, these are the same benefits we would suggest lay before any team looking to try Scrum.

Isn’t this the good that Voltaire would caution us against passing on?

Larman and Vodde have some great advice about how to go about ever more closely approaching the “potentially shippable product increment” goal of the Sprint through expanding the Definition of Done (DoD below):

“In general, these are the ways of expanding the DoD:

• Automate – for example, performance testing is automated
• Expand team cross-functionality – for example, a person with technical-writing skills joins the team”

That latter idea suggests a path to improvement that starts in development and then spreads over time to the remaining functions.  If we view Scrum deployment as being something we do in the context of projects and products, this is both natural and reasonable.  But, if we view deploying Scrum as something we do in the context of teams of people or if we view it simply as “transforming the world of work,” then why would we believe we have to start with any particular set of people?  Why not start with testing and grow our way towards development?

Would that be a half-assed approach to deploying Scrum?  Perhaps, but like Richard Dawkins’ half a wing or half an eye, maybe half an ass may prove a more useful incremental improvement than may be apparent at first glance.

So, can you deploy Scrum to a test team?

Sure, why not?

What do you think?

About the Author: Ed Willis has been a ScrumMaster, Product Owner, Scrum coach and trainer.  He is currently a developer in the telecommunications industry.

This week both Jean and I delivered talks on the Agile organization at Agile 2010 in Orlando. Whether you were able to attend one, both or neither, this post shares the handouts and materials that we used in the talks.

If you attended, please provide comments on what you liked, were puzzled by and might change in the future.

Jean’s work was a three-hour tutorial on learning models for managing the Agile organization.   She ran three exercises and provided a bibliography of books/resources that we have used here at Rally:

Jean in action at Agile 2010

• True North Example

• True North Exercise

• Strategy A3 Template

• ORID Exercise

• Personal and Corporate Visioning Exercise

• Sample Quarterly Agenda

• Bibliography

In addition to Jean’s talk, I presented an experience report on our use of Plan-Do-Check-Act (PDCA) at Rally.  This report tells a story of our evolution of strategy execution from Gazelles/Scrum to Lean/Agile.

We hope these resources provide you with ideas for scaling your own Agile efforts beyond their current levels.  Again, please comment on the blog with what you got from the materials or the talks.  We want to hear from you on this topic.

Ryan Martens is a tomato grower, founding board member of the Entrepreneurs Foundation of Colorado, and CTO at Rally Software Development.

There are other conferences that cover Agile software development, but the Agile 20xx show reigns supreme. At nearly 2000 attendees from around the world, this year’s show is happening at Walt Disney World in Orlando.  (It was moved there after the flood in Nashville.) For the first time, three of the major analyst firms (and as a result 5 of  the key analysts who cover Agile and ALM) are attending the conference – Forrester, Gartner and IDC.

Rally coaches, sales and marketing folk at booth setup

As a result of the show’s success, it has become the most significant market rhythm in our industry.  So this week, we announced a few things:

• Rally Software Extends Agile ALM Platform to Meet the Unique Needs of Global Organizations – New customization options, full traceability with build and source code, and sharing with corporate dashboards create a single source of record for progress and quality
• Rally Software Extends Visualization of AgileZen Kanban Board – New non-profit program offers a 50% discount on leading visual project collaboration tool
• Innovation Games Company Announces Integration with Rally Software – The leader in serious games for business brings customers improved collaboration and prioritization capabilities
• Tasktop Introduces Agile Planner for Eclipse – New product enables Agile planning from the IDE and integrates multi-vendor Agile ALM solutions

I am speaking tomorrow on PDCA: Moving Beyond Simple Inspect and Adapt. (Thurs 9:00 a.m. in A-1). Other Rally speakers remaining this week are:

Get your Rally cap

• Former Rally developer turned Rally technical account manager turned Rally coach Chris Browne speaks Wednesday on The Art of the Hackathon (Weds 15:30 – 17:00 in Asia 3).
• Rally coaches Alan Atlas and John Martin speak Thursday on “Your Team, Your Freedom, Your Responsibility” (Thurs 15:30-17:00 in Asia 3).

Follow the news from the show on Twitter at #Agile2010. Come see us at the booth and get a Rally and Deliver ballcap. Or let us know if you’re not at the show and want us to send you one (send name and address to kcaraway@rallydev.com).

Ryan Martens is an organic farmer, founding board member of the Entrepreneurs Foundation of Colorado, and CTO at Rally Software Development.

I have a reason for liking Bob Payne. Bob has empathy and a true love for giving back. That resonates with some of what we are trying to do here in Boulder. Rally, as a B Corporation, has expressly created a charter about giving back to the community: 1% equity giveback, 1% employee volunteer hours (over 2500/year in the last two years) and a number of other local not for profit initiatives. For Bob and us, adopting Agile has  been an important component in how will pull our empathy and our software skills together. With Agile, we seek to deliver feasible, effective, desirable solutions in our complex world. And reaching beyond our corporate walls to deliver that desirability catapults us to being truly empathic in our solutions.

When you meet Bob, you immediately get what “giving back” and empathy is about in his Agile work and beyond. Bob is always looking for new ways to bring Agile to our community and the greater community: our complex world. Out of his own interest in giving back to the Agile community, Bob set up his Agiletoolkit podcasts site. A gift for all of us. At the recent ADP West conference, Bob was there with his sound setup.  Bob took interest in Rally’s Agile Zen acquisition when interviewing Ryan Martens. And I  had the great fun of talking about Seth Godin’s book “Linchpin” that both Bob and I had read.

In this post, I’m so honored to have the opportunity to turn the tables on Bob and be the interviewer.

“Bob, what got you started recording your Agiletoolkit podcasts?”

I began recording the Agiletoolkit podcasts in 2005 after hearing several interesting podcasts and wondering if anyone would be interested in a podcast about Agile. I had always been a gadget person so fiddling with recording equipment and microphones was a natural for me. In fact, I now also have an iPhone App for the podcasts.

I love having the conversations and the podcast gave me an excuse/push to have conversations with people that I might not connect with in the halls at a conference.  A good example of that was when someone said to me, “You have to talk to this guy Arlo.” Without that introduction via the podcast I am sure I would not know Arlo Belshee as well as I do now.

While I am by nature gregarious, I do not search out “networking opportunities”. The podcasts have forced me into a new comfort zone that includes a lot more people from the community than I would have connected with through normal channels.  While I hope people appreciate and benefit from the podcasts, I do them for myself.  That affects the style of the podcasts. Since I am not trying to be polished or create an edited product, the podcasts have a more natural/comfortable feel.  I just wish I said “UM” less and a was little more polished on my delivery. But…I am who I am and it is what it is.

“How did you get into Agile philanthropy?”

Agile philanthropy started as a way of trying to meld my passion for doing good in the world with my passion for agile methods.  Using the power that is evident in the agile community to do great things is one of the goals of Agile Philanthropy.  Ideally we will get to the point that this movement is self-sustaining. But we are really just starting out on this journey.  I hope that I can grow the movement in the direction of local chapters doing work for local not for profits. Right now everyone is very busy and I am the bottleneck.  We are currently working with Mano a Mano and Haiti Partners. And, I would love to have people with a passion for a particular cause to contact me and start up their own chapter.

“What about your other philanthropic interests?”

I am very interested in local sustainable food, economic development and social justice. I volunteer in my kids’ schools quite a bit.  Most recently, I built incubators with the kids and hatched chickens and worked with the teachers to incorporate that into the curriculum. I have been working to get local food into the schools; to create school gardens; and, to relax the laws in Washington DC as they pertain to the keeping of bees and hens. Most of my other work is more directly related to the work I do in Agile Philanthropy.

“When did you start the Mano a Mano project work and what have you and your yearly teams accomplished at the Agile conferences?”

Seems like forever but we introduced Mano a Mano three years ago when the conference was in DC.  I was running the development lab in the basement and hoped that I could get some real work done in the lab that would do some good.  After that, I tried to make it more formal and improve what we have done for them each year.  They have been very appreciative and very patient with us since I am learning as I go with this process.

To date, we have moved them onto a Content Management Platform and developed their iPhone optimized donation page.  Most importantly, I am happy that I have connected Mano a Mano with David Hussman and a number of other volunteers in the Twin Cities that are helping out on a regular basis.  Wayne Simacek showed up for an event that Jeff Patton and Ed Kraay were holding to help Mano a Mano define their web strategy and ended up staying on as a volunteer member of their IT staff.

It is that kind of leverage that I hope to bring by connecting the two communities.

“What do you have in store for us at the Agile2010 conference?”

For the Agile2010 Conference, I am working again with the UX stage to do an Extreme Makeover for the Mano a Mano web presence.  We hope to be able to work on their information architecture and site design to improve the impact of the message that Mano a Mano is putting out. We are looking for volunteers to come by the LiveAid lab and help with the effort (hint, hint).

I also hope to get people interested in replicating this model for not for profits that they are passionate about.

You can do this too

To end this post, I want to thank Bob for the example he sets for all of us. I also want to emphasize Bob’s call to action to get engaged locally. You can do this through your existing local Agile group. Or, you can create a new group with an express charter to give back to the community. Recently Brad Feld here in Boulder wrote about the “Boulder New Technology Meetup” event that supported over 300 people engaged with 20 local non-profits. And here at Rally, we are marching along with Bob philanthopically working to give back: supporting  Intercambrio, donating time to local non-profits (Community Food Share and Growing Gardens) and working with the Salesforce Foundation.