Agile Development Blog

I just played our fourth Agile portfolio planning game with a team of executives. First they had to rank and plan a portfolio of new product ideas, enhancements, architectural refactorings, and other work. It’s tough to balance competing priorities against strategic financial targets. Then we simulate a monthly steering council that must react to new learnings, development mishaps and market changes, all while maintaining strategic alignment.

I’ll give you a hint to winning the game:  Turn that waterfall team into Scrum teams early, and pay attention to quality.

Wagged by Plans: Is It Time to Change?

Do any of these problems sound familiar?

Execs play an Agile steering game

• The organization funds all or most proposed projects, and expects to make progress on them all simultaneously.
• Teams adjust each iteration based on new learnings, but the annual plan for the organization does not change.
• Teams adjust, but it’s hard to know whether the adjustments throw the portfolio off-goals.
• Projects are managed within silos; trade-off decisions are not made across the portfolio. As a result, some silos are under-capacity and others are overcapacity. Or, the whole doesn’t add up to the strategic objects for the organization.
• Teams are judged on their ability to deliver value; the portfolio is judged on its ability to meet plan.

If these scenarios apply to you, you’re probably ready to bring Lean and Agile principles into your portfolio management process. Especially if you have already solved these problems:

• Your Scrum teams are cross-functional and persistent and have learned to deliver consistently on iteration and mid-range commitments.
• Your Kanban teams have reduced significant waste in their value stream and consistently deliver to service level agreements.
• Agile teams have achieved a fairly consistent velocity/throughput (amount of work delivered per timebox) while delivering against quality goals.
• The Product Owner group elaborates requirements just-in-time, feeding teams a steady stream of high-value work.
• You have added the structures that enable multi-team programs to coordinate dependencies and deliverables within a closely related group of teams.

In working with earlyvangelists to develop our Agile Portfolio Management solution, we learned that in most organizations the portfolio is managed under the old, plan-driven paradigm, which is at odds with the Agile practices at the team and program level. However, it is equally clear that we cannot begin to challenge that paradigm before we’ve reached a certain level of Agile maturity.

It is too hard to engage the business before development has a track record of success.

It is too hard to build a realistic roadmap before teams have developed a steady velocity.

It is too hard to emphasize steering before teams have the discipline to deliver iteratively and incrementally.

Advance Your Agile Practices: The Agile Enterprise
Our early work delivering the Agile Portfolio Steering service has been exciting.  Fellow coach Isaac Montgomery and I have watched a room simply buzz as a group of leaders begins to imagine and employ the paradigm shift from plan-driven to value-driven portfolio management.

The buzz begins because of the people in the room; namely, a cross-functional group of leaders (directors, VPs, SVPs) from the business, product/portfolio, and technology. Together, we explore and understand the current challenges around building realistic roadmaps, tracking the progress of initiatives, and leveraging the agility of teams without losing focus and alignment. Sometimes this group has never built so shared a view of the organization.

There can be a moment of frustration when we’ve made the challenges so clear. But then the energy gets high as Isaac and I begin to provide possible solutions in the form of Lean practices. Together we dig into redesign their portfolio processes, to inject the visibility, feedback, governance and focus that enables Agile to work at scale.

Rally has already begun to see patterns in the Lean practices that implement more adaptive portfolio management.

• Map the value stream from concept to cash, and build a Kanban board to represent those states. “In Development” is just one state.
• Limit work-in-progress on that board. Fewer initiatives in play means better throughput, productivity, time to market, responsiveness, and reduced risk.
• Emphasize value and budget over cost estimates. This requires new data and a new mentality.
• Provide clear strategic objectives  and constraints, and then let the people closest to the work (the teams) plan the work. Use clear visualizations and data to verify the results align back to the strategy. Help teams adjust as necessary to remain aligned.
• A rolling wave cadence of planning and steering helps your leadership be both strategic and adaptive.
• Effective portfolio management will require planning and learning at different levels of the organization, on different levels of abstraction and strategy, and on different cadences.

The Service: Re-Tool for Agile Portfolio Management
Built on those early deliveries, I’m excited to now offer more generally our Agile Portfolio Steering workshop. I think it provides a powerful benefit through focused work.

Inspired by the lessons of the portfolio planning game, the leadership group works with an Agile coach to begin to redesign its portfolio management process:

• How could we have better value data on proposed projects?
• How can we decide our investments rather than be wagged by cost projections?
• How could we fund fewer projects?
• How could we commit to goals rather than to lists of features?
• What data would we need to be able to steer the portfolio effectively?
• How often should we steer the portfolio? A project? A program?
• Could we reorganize our teams to enhance flexibility?

The outcome might be a new planning process, new data requirements, new progress visualizations, new planning and steering councils, and more.

The service is listed as “2 days”. In reality, we have learned that a leadership group might not spend two whole days in a room with us, but the hours they commit will be intense, with short breaks and working lunches, because this work matters. Dollars are at stake; business success is at stake. We will work together to design the right on-site session.

Stay tuned for more details during our Dec 6 and beyond launch events. We look forward to working with your organization, to understanding your context, and what challenges you have to align the work of your Agile teams and your strategic goals. What prevents you from delivering all the value you could to your business?

As Rally’s “Solutions Evangelist”, Ronica Roth promote solutions for our customers that combine product and process. She also promotes fun that involves mountains and snow.

In our Agile Tooling for High Assurance Software Development series we’ve looked at a prescribed user story structure for requirements traceability and we dove into how tooling is important to maintain relationships needed for traceability to other artifacts in the system (defects, test cases, etc.). Given this, how do we automate a traceability matrix from Rally?

First let’s review the why. Why do we spend so much time and energy creating traceability matrices?

There are a number of sources that I could cite but I’ll go back to the Design Control Guidance for Medical Device Manufacturers, where section F Design Verification states:

DOCUMENTATION OF VERIFICATION ACTIVITIES. Some verification methods result in a document by their nature…

Another self-documenting verification method is the traceability matrix. This method is particularly useful when the design input and output are both documents; it also has great utility in software development. In the most common form of the traceability matrix, the input requirements are enumerated in a table, and references are provided to each section in the output documents (or software modules) which address or satisfy each input requirement. The matrix can also be constructed “backwards,” listing each feature in the design output and tracing which input requirement bears on that feature. This reverse approach is especially useful for detecting hidden assumptions. Hidden assumptions are dangerous because they often lead to overdesign, adding unnecessary cost and complexity to the design. In other cases, hidden assumptions turn out to be undocumented design input requirements which, once exposed, can be properly tracked and verified.

Although this is only a guidance, it is pretty clear that the FDA is encouraging the building of traceability matrices, that might be “why” we all create them. In order to automate the creation of this matrix Rally has offered two Rally Catalog apps in the product, the Iteration Traceability App and the Release Traceability App. The screenshot below shows an example of the Iteration Traceability App that displays Stories with their associated Tasks, Test Cases, and Defects for an Iteration.

In the spirit of the work that Dean Leffingwell and I have been doing I persuaded one of Rally’s awesome App Developers, Charles Ferentchak, to help me build an additional trace matrix app based on an example shared by a customer. This app is not bound to an Iteration but instead allows the creator to select from their list of tagged PRD’s and creates a trace report showing all of the features associated to the PRD along with test cases, results, design outputs, and etc.

Both of these traceability matrices are examples of possible automation that can be easily achieved by adding tooling to your Agile practice.

To submit your own idea for a Traceability Matrix to Rally’s Product Team visit Rally Ideas for Regulated Industries and the next time you are on LinkedIn be sure to join the Agile in Regulated Environments LinkedIn Group.

And, as always, I encourage your comments on this post or email craig@rallydev.com.

Craig Langenfeld is a Technical Account Manager with Rally Software Development who has been working closely with many Agile Practitioners in the Healthcare Industry to define best practices.

Dean Leffingwell is an entrepreneur, executive, author and consulting methodologist who provides agile transformation consulting services to large software enterprises. Between consulting engagements and keeping up with his Blog Dean managed to publish his latest book, Agile Software Requirements.

Are you challenged with trying to map Agile Software Requirements to a traditional requirements model for the documentation required during internal or external audits?

If you answered “yes”, you are not unlike many of Rally’s highly regulated customers who want to reap the benefits of Agile but struggle to find good tools for creating and managing system documentation. For the next two posts in our Agile Tooling for High Assurance Software series, I will provide an overview of the material that I presented to a small sampling of customers in early May at our inaugural RallyON conference in Boulder. This work is a continuation of the project that Dean Leffingwell and I have been working on over the past year.

Let’s start by reviewing Dean’s post, Agile Software Requirements in High Assurance Development, which provides us with a traceability model as well as a medical device example to follow. From there I’ll demonstrate how Rally can be used to support traceability from what we traditionally think of as a PRD, a document detailing the system’s intent, to user stories that are commonly used to implement the features of the system.

An important part of the verification effort is to assure that every feature traces to one or more user stories (or other forms of software requirements expression) as is illustrated below. But first we need to begin with a PRD that defines the overall behavior and intent of the system. This will serve as design input in FDA terms. The PRD is going to be decomposed into individual features that will be implemented by many Software Systems Requirements, which we’ll also refer to as User Stories. Take a moment to review the illustration.

With regard to managing these requirements, many of us have encountered or even created spreadsheets with thousands of rows of requirements and tens of columns of attributes that described the system we intend on building.

As part of the introduction to Agile practices we will trade-in the spreadsheet for an Agile Lifecycle Management (ALM) tool that will provide us with better traceability to the real system being implemented. An ALM tool, like Rally, allows us to create a high level user story that will represent system intent and maintain those items that we said were important to document such as a PRD. In Rally the system, intent, features, and user stories will all be tightly coupled by creating a family of user stories through parent/child relationships.

The screenshot below shows the EPAT example that Dean Leffingwell described in his post, extrapolated in Rally. Notice that our EPAT Device Enhancement has been broken down in a way that will allow us to trace each child user story back to the PRD (represented by a parent user story). Another convention being used in this example is that of tagging. Rally users often tag stories to represent requirement types like PRD, SRS, functional, or non-functional stories for better reporting on specific story groups.

Being prescriptive and consistent when structuring our Agile Software Requirements (User Stories) in Rally provides us with the unique capability to create our own custom document exports, such as the Product Requirement Document that I will highlight in a post next week.

In summary, Rally helps us to transform individual user stories that each implement a small increment of value into a aggregation of stories that represents the intended product.

To learn more about Rally’s User Story Hierarchy watch this informational video.

To submit your own ideas to Rally’s Product Team see Rally Ideas for Regulated Industries and the next time you are on LinkedIn be sure to join the Agile in Regulated Environments LinkedIn Group.

And, as always, I encourage your comments on this post or email craig@rallydev.com.

Craig Langenfeld is a Technical Account Manager with Rally Software Development who has been working closely with many Agile practitioners in the healthcare industry to define best practices.

Dean Leffingwell is an entrepreneur, executive, author and consulting methodologist who provides agile transformation consulting services to large software enterprises. Between consulting engagements and keeping up with his Blog Dean managed to publish his latest book, Agile Software Requirements.

This is the second post in our blog series, Measuring the Impact of your Agile Investments. The series focuses on measuring the impact that Agile practices have on business outcomes.

For many leaders, increasing the productivity of the development organization (delivering features the business needs and generating improved ROI) is their primary and over-riding goal. ‘Doing more with less’ is a mantra they preach to their teams continuously, and which colors every decision they make.

Yet few have a clear and consistent definition of productivity or an effective means of measuring it.

Productivity: a relative measure of the efficiency of a system in converting inputs into useful outputs.

Productivity: the ratio of the real value of outputs to the combined input of labour and capital.

Productivity: a measure relating a quantity and quality of output to the inputs required to produce it.

So, in simplest terms:

Productivity = Valuable Outputs / Costly Inputs

Be Careful What you Wish For

While I am a ardent believer in the value of metrics, it is important to keep in mind the potential for unintended consequences. Individual and group behaviors will evolve to meet your measures – not necessarily your goals. So ensure that the measures you use promote the behaviors you desire.

Most documented productivity metrics in software development define a unit of output as either a KLOC (thousand lines of code) or as a Function Point. Both have significant disadvantages.

KLOCs are generally favored for the simple reason that they’re unambiguous and easy to measure. However, they can unintentionally promote poor design and coding practices. Elegant, efficient, maintainable software takes more time to write, and takes fewer lines of code. Therefore, a KLOC metric inadvertently creates a disincentive for building high-quality software, and rewards poorly thought through designs and shoddy craftsmanship.

While Function Points don’t necessarily promote poor design and craftsmanship, they have their own unique challenge – they are famously difficult and expensive to calculate and measure.

Perhaps even more importantly, both KLOCs and Function Points ignore the core aspect of ‘Value’ inherent in our definition of output – and therefore productivity.

It has been stated that 64% of the features and function in the typical software system are rarely or never used (Standish 2002). Calling the code that delivers these features and functions “productive” may be a mischaracterization.

Similarly, those features and functions that ARE commonly used are generally not of uniform value. The Pareto Principle would suggest that 80% of the value is delivered by 20% of the effort. Yet KLOC and Function Point-based metrics treat all features and functions (and code) delivered as interchangeable. This promotes focusing on the easiest and lowest technical risk; rather than the most valuable, most innovative and greatest business risk…

Output is a measure of the value delivered, not the effort expended.

A number of people in the Agile community have written about an alternative unit of output measure – Value Points.  While the simplicity and value focus of this model is appealing, it has challenges when scaled beyond a few teams. In order to be meaningful at the organizational level, you must normalize the relative value point scale across teams and programs – which can be difficult and expensive.

Also, the Value Point approach does not easily translate to initiatives and/or divisions that may not be delivering in an Agile manner. Having a common measure of output, and therefore productivity, is critical to measuring the impact of your Agile investment.

So, the approach we recommend is to associate a percentage of the total value of a given initiative to each Minimally Marketable Feature (MMF) or production release. These percentages can then be applied to a any monetary business justifications (ROI, NPV, Discounted Cashflow, etc.) to arrive at an expected dollar value realized from each release.

Hence,

Productivity = Total Value Realized (delivered to production) /  Total Cost of production (labor)

Using this approach, your organization (be it a Scrum team, a multi-team Agile program, a waterfall project, or even an entire product development group) base-lines their productivity for a period of time and monitors the change over time.

Example:

The division has 3 initiatives in progress:

• Initiative A:  Total Expected Value $5MM; being delivered by a Scrum team with an iteration run-rate of $70,000.
• Initiative B:  Total Expected Value of $25MM; being delivered by 5 Scrum teams with a combined iteration run-rate of $400,000.
• Initiative C:  Total Expected Value of $50MM; being delivered according to a project plan and resource matrix that charges $2.5MM to the project in the 1st quarter.

In Q1:

• Initiative A: Released to production monthly and delivered a total of 60% of Expected Value; or $3MM. 25% of the backlog has been burned-down in terms of story points. Total Cost $210,000. PRODUCTIVITY = 14.29
• Initiative B: Released to production quarterly and delivered a total of 65% of Expected Value; or $16.25MM. 35% of the backlog has been burned-down in terms of story points. Total Cost $1.2MM. PRODUCTIVITY = 13.54
• Initiative C: Completed requirements definition and is 50% done with Detailed Design, and delivered 0% of Total Expected Value. Total Cost $1.2MM. PRODUCTIVITY = 0

*undelivered work is WIP, and therefore not yet productive.

Aggregate Division Productivity for Q1 = 7.38

As you can see, it would be relatively straightforward to predict Q2 productivity – at the initiative as well as division levels – by assessing the various product roadmaps and traditional project plans. Those projections could then be used to:

• drive discussion about trade-offs on where to allocate limited capacity and maximize productivity
• staff and fund initiatives where productive potential is high, and to cancel successful projects whose greatest productive potential has already been harvested
• inform intelligent business decisions – which is WHY we’re measure outcomes

By understanding the productivity of the development organization (its efficiency in delivering features the business needs and delivering improved ROI), leaders can effectively drive improvements and business decisions that maximize productivity – Doing More with Less.

The next topic I’ll address in the Measuring Impact series is Quality. Most organizations claim to have a Quality focus, yet few take a holistic view of Quality or appreciate the strong correlation between Quality and our other Outcome Metrics – Productivity, Responsiveness, Customer Satisfaction, Employee Satisfaction and Predictability.

How are you measuring the impact of your agile investments? Please share your ideas and feedback in the comments.

Isaac Montgomery is the harried father of twin sons, the adoring husband of Angie, a frustrated hack on the golf course, and an Agile Coach at Rally Software. He blogs at Leading Results, you can follow him on twitter at @iwmontgomery

Welcome to our new blog series, Measuring the Impact of your Agile Investments. This series focuses on measuring and quantifying the impact that Agile practices have on business outcomes. We’ll explore how putting comparable metrics in place allows organizations to steer improvement efforts in the right direction and realize the biggest payoffs of Agile.

“How do we measure the success of agile?”

It’s one of the most common questions we hear from senior leaders.  And it’s a critically important one for agile evangelists working to justify the organization’s investment in agile and maintain momentum as other priorities compete for leadership attention. The agile community’s typical response to this question has been some form of an agile maturity assessment – such as the Nokia Test. These tools are clear, easy to use and can be extremely effective in helping organizations assess their adherence to good agile practices.  Yet, when used in isolation, they can leave senior leaders unfulfilled – and miss an important opportunity for aligning agile to fundamental objectives.

For business leaders, the question isn’t how well are we doing agile?  The question is how well is agile doing for us?  What impact is agile having on business results? These are the questions senior leaders really want answered.

My response has typically been: “How did you measure your impact on business results before Agile?”

Which is generally met with awkward silence and a muttered admission of: “Not very well.”

The conversation then moves toward why measurement is expensive, why you can’t show progress if you don’t have a baseline, and why you need to be very careful about what and how you measure – lest you create unhealthy behaviors and unintended consequences. Someone ultimately quotes Einstein, we nod our heads thoughtfully, and finally move on to other topics. Crisis averted!

But the question remains: “How do we measure the success of agile?”

And, if Agile Success = Business Success, then the real question is: “How do we measure business success?”

Which is the question we’re beginning this blog series to address.

While every organization will have their own unique objectives and priorities, most can be encapsulated as some combination of these:

• Productivity
• Quality
• Responsiveness
• Customer Satisfaction
• Employee Satisfaction
• Predictability

In the posts to follow we will examine each of these Business Outcomes and look at:

• How can we measure the business outcome?
• What agile practices are effective levers in improving the business outcome?
• How can we measure our agile levers as leading indicators of improved business outcomes?

The first topic I’ll address in the Measuring Results series is Productivity, because for many leaders, increasing the productivity of the development organization is their primary and overriding goal.

Isaac Montgomery is the harried father of twin sons, the adoring husband of Angie, a frustrated hack on the golf course, and an Agile Coach at Rally Software.  He blogs at Leading Results, you can follow him on twitter at @iwmontgomery

Does your current tooling environment provide traceability paths for User Story Verification within an Agile Iteration? My last two posts in this series described how Agile tools, like Rally and Quality Center, are used to achieve this very objective. In this post I will bring the User Story Verification conversation to a close by elaborating on the last two points of Dean Leffingwell’s proposed Traceability Paths for High Assurance Software Development.  The final two points are:

• User story to code – path to the SCM record that illustrates when and where the code was changed to achieve the story
• User story to code-level unit test (the “white box” test which assures that the new code works to its internal specifications, also maintained via path to the SCM change set)

Dean provides us with a nice illustration of the Traceability Paths Model to review before we hop into a tooling example:

Let’s start with User Story to Code!

(more…)

Ooooh! It’s Friday and time for another celebration of a one hit wonder. And though it is April Fool’s Day, this is no joke. Today I am thinking about how we have to continually hone our craft in our technology world. Sometimes, I feel as though I am in a never ending PhD program. Or, I keep moving from one apprenticeship to another. It can be overwhelming. And yet, I love my work. I love feeling growth and challenge in how we continuously improve the software industry.

I started in this technical world 30 years ago, programming JCL on punch cards. It was that or paper tape. (I am NOT making this up!) I moved on to PL/1, Fortran, C, and a couple of assembly languages along the way. I then made the dramatic move from procedural languages to object-oriented languages.

And there were other areas in which the world of technology invited my growth through challenge and reward. I slowly moved away from a very documentation-driven view of the world. (Well, I could call it “graduated from” but I don’t want to be too provocative on a “One Hit Wonder Friday!”) And there was a time when my sense of significance was deeply wrapped up in my prowess in Microsoft Project, in 5-level work breakdown structures, and in pages of perfectly aligned Gantt charts based on my skills with Finish-Start dependencies. Whoa! Just had a trip down memory lane! It’s a heart-breaking story. And it is worth telling. Because, now I am in an Agile world.

Today, I am pausing and reflecting on how I have had to continually be prepared to re-tool my thinking and my tools. It hasn’t always been easy. And, it has ALWAYS been rewarding. Moving to the world of Agile and continuously working on my grasp of lean thinking, systems thinking, complexity theory and knowledgement management has all been a challenge. The rewards, however, are priceless.

And now our one hit wonder of the day. This week I am thinking about an artist not from the music industry. His work was limited to the big (and little) screen. He was brilliant in his craft at the time of his fame. In fact, I would say he was singular, outstanding, ground-breaking. He won the hearts of millions. And yet, he fell into obsolescense to the point of later appearing in television shows and movies more as a perfect caricature of himself. Why? He didn’t keep up with technology. He remained in a world in which his craft lost its value. As good as he was, he was left behind. He is a poignant reminder that we must always invite growth and change even when we may feel lost or that it is all just too much. Our artist for today gave selflessly to others when all odds were against them.

Yes, today we are celebrating “Robbie the Robot” from the the 1956 movie “Forbidden Planet”. The spelling of his name (Robby/Robbie) seems to be somewhat of a mystery just as Robbie’s dedication to his performance was mystical. Though he appeared in a variety of movies and TV shows after “Forbidden Plant” (including “Lost in Space” no less), these were only as brief cameos, as sad reminders of glory days long past.

I met Robbie several weeks ago in the Intel Museum in Hillsboro, Oregon. I was at the Intel Agile Conference there when Scott Hanselman of Microsoft asked to interview me for channel9.msdn.com. Where better to hold the interview than in the museum just across the way from the conference? And there he was, Robbie the Robot. Wow. Me meeting Robbie the Robot. Robbie still holds up his head with pride despite being relegated to museum status. He still holds out his large rounded, clumsy digits ever ready to offer assistance despite his inability to do so. I salute Robbie for all he has given us.

Jean Tabaka is a crash skier, college hoops shredologist, author and Agile Fellow at Rally Software Development. You can follow Jean on Twitter at @jeantabaka

With the help of a large and passionate community, Dean Leffingwell and I feel confident in answering yes to the question we posed last December, “Is Agile the key to building high assurance software?” Based on the feedback we’ve received in Dean’s Scaling Software Agility Blog and the large number of medical device companies already successfully utilizing Agile practices, it’s clear that Agile is vital to bringing highly innovative and high assurance products to market before your competitors. Here are a some of the comments we’ve received from practitioners.

“I think this is quite interesting, and quite possibly could revolutionize the way medical device software is delivered.  Life sciences is the last bastion of waterfall development methodology. I look forward to championing the cause!” - Bernie Barbour

“Having been in FDA regulated development environments for over 10 years, this truly is an area where most people live & breath waterfall. But as you said, there are a few out there already doing agile in this domain and I would argue that it can be done (relatively) easy.” – Michael Meissner

The community really stepped up to help us pave the way for developing the first guidance truly focused on the details of implementing Agile in High Assurance Environments using Medical Devices as are exemplar (see topics like: Software V&V: Verification Testing Features and The Validation Sprint Explained). Having presented the meat of the process in our first series, we are excited to move forward on a second blog series, Agile Tooling for High Assurance Software Development.

In this series, Dean and I will provide examples of tools that many of us have in our own environment, such as Rally, Quality Center, and SVN, and show how these products can be used to support the compliance documentation that is commonly produced for audits in regulated environments. We will provide the community with best practices for using Rally as well as a best of breed solution that integrates Rally to other products.

The series will include posts like…

1) Use Rally to track and report on the basics – Define | Verify | Build
2) Achieve tight traceability and reporting from Product Requirements to System Requirement to Verification & Validation
3) Don’t overlook the implementation of the Validation Sprint
4) Testing and test traceability in High Assurance Environments

In conjunction with this series, Rally will release new applications like an improved traceability matrix and a SRS report that Rally customers will be able to take advantage of. In addition, we encourage you to share your suggestions for reporting that would help to automate the compliance process for your Agile team. If you are a Rally user, please enter your ideas at Rally Ideas and tag it as “compliance.” Or, feel free to send me an email at craig@rallydev.com.

If this topic interests you please leave us a comment below, on Dean’s Blog, or send me an email.