Recently, my Rally colleagues Ben Carey and Ryan Martens delivered a great webinar about testing and quality. What particularly struck me about the session was how Ben set up why we should, at a very personal level, care about testing and quality.
Enter Amy and her daughter Morgan.
As Ben told the story, Amy was a back office admin in a physician’s office. It was her responsibility to get billing out to insurance companies for the practice, and on one particular day, things were not going well.
She kept getting cryptic error messages. The batch just wouldn’t go through. Stress mounted. To add to Amy’s distress, it was nearing time for her to pick up her daughter Morgan at school. Morgan had just started school. Only 4 weeks in, Morgan was suffering some separation anxiety. Amy felt the urgency to be there for Morgan as soon as school let out. And yet, she was stuck at her desk dealing with these insurance issues.
Why should Ben be talking about Amy and Morgan in a webinar about testing and quality? Because Amy was Ben’s customer. Ben was on-site to see how his software was supporting the practice. And so, he personally felt Amy’s mounting stress in using the software HIS team had delivered and HER doctor’s office had paid for. When Amy started to cry, it was all over for Ben. He had to do something.
What does Amy and Morgan’s story tell all of us?
1. Users deserve better than this.
2. On an Agile team, quality is everyone’s job!
We often think of testing as an issue for the tester role alone. But this stance sets a number of bad dynamics in motion:
When we don’t fix defects, we declare doneness by “hiding” defects in a defect backlog
Hidden defects accumulate into hidden technical debt
Technical debt slows down our ability to deliver subsequent releases
An ever-increasing defect backlog can be demoralizing for the entire team
Engineering resents the business when being forced to deliver features when the engineers know defects exist
Business resents engineering for not being better and faster at building features
Testers resent testing left until the end because it puts business, engineering, and testing in a tight spot
So here is a thought: quality is not just an engineering issue; it is a systemic issue. That is, it is driven from the top down in a business not from within one part of the organization. A business policy about quality impacts our users by directly impacting their quality of life. Amy’s stress about picking up Morgan from school could be traced back to a testing and quality policy in engineering that was driven by a policy in the business: get features out.
As you adopt Agile, work vigorously to create rigor in your testing and quality goals. Why? Because your policy can hit hard in ways you don’t currently track. Pay attention to more than just the internal cost of technical debt. Let’s pay attention to the quality of life of our users.
The next time you are logging a defect versus fixing it, don’t forget about your Amy and Morgan.
About the Author: Jean Tabakais a wine enthusiast, author and Agile Fellow at Rally Software Development. Subscribe today to get free updates by emailor RSS.
In this post, I’d like to respond to some questions posed to Johnny and me during the webinar.
JET = Jean Tabaka
What is the measurement of productivity? Is it user stories per lines of code?
JET: We define productivity index with regard to the Michael Mah statistics collected over 7500 products. It is not about lines of code. Rather it is about delivered value. At a very high-level view then, a PI in an Agile team is about the amount of value delivered by a team based on value delivered over the total number of team members.
Michael Mah shows in his studies, which we referenced in our webinar, that Agile teams tend to have a higher ability to delivery value per team member than the norm across all projects he has surveyed. This supports our view that the way to project cut costs is to help team members increase their PI versus cutting team members or off-shoring work.
When you start on Step 1, one team in flow, do you start immediately with 2-week iterations, even if the teams are used to waterfall?
JET: For any team starting its transition to Agile, the important Flow guidance I give is: Choose a timebox, and stick to it. Religiously.
I give guidance that a shorter timebox is preferable to a longer one. This gives the team that moment of pause for inspection and adaption sooner rather than later.
A shorter timebox invites earlier improvements on process. It also affords the Product Owner or customer representative the opportunity to re-prioritize, add, or delete items in their backlog.
Given these parameters, the team commits to what the timebox length will be. So what do I advise? If asked, I tell teams that 2 weeks is a good starting point: shorter time to inspection versus longer wait for improvement opportunities.
How would you apply Agile, and particularly Scrum, to security related software where design and reviews typically take longer than one iteration?
JET: Agile asks us to evaluate how much design and review MUST occur before features can begin to be coded and tested. Your question is all the more important in your context.
Think of it this way: Imagine in a security software environment, you implemented something and then slammed all kinds of vulnerability tests against it to discover its actual vulnerability. Imagine that this could be in addition to, or even as a replacement to, extensive review up front.
I once worked in a security related software environment. We spent months and months reviewing documentation attempting to conduct independent validation and verification about vulnerabilities before any coding began. How I wish we could have just conducted detailed design, code, and test on our riskiest features and applied our hypotheses against these rather than being left to hope our analysis was adequate!
If we want to introduce Agile in a certain project team, what’s the perfect ratio of Agile consultants versus our own personnel?
JET: Perfection is in the eye of the beholder ;-)
I suggest one mentor per team initially. We have also used one mentor for a team of teams, a program. That creates stability and synergy across the teams and encourages all teams to form their own Agile team agreements while also learning from what has worked well with the other teams.
One more thing: Having a mentor at the management/executive-level is also critical. It may be this same person; it may be someone who has more of an executive stature. In the latter case, be sure the two mentors work tightly with one another.
Would you define ‘user stories’? Similar to Use Cases? Difference?
JET: User stories are an expression of a request from the customer or customer representative for a particular function. They are meant to be short, incomplete, and value-based.
They invite conversation and further clarification of completion: “As a < role >, I want to < action > so that I can < benefit >.”
“As a dog, I want access to ‘The Daily Woof” newspaper on my Kindle so that I can flip pages without opposable thumbs.”
This request for action and benefit is elaborated as it rises in rank among all functions requested. We elaborate these in several ways. One way is to start writing acceptance criteria.
Another way might be to elaborate it through a use case that details actor-system interactions, extensions, and so on. A team makes agreements about how function requests grow in clarity; that is, are use cases our best way to declare DONENESS for this request? Or, in our environment, is a simple description and test case sufficient?
How can we get better reporting metrics for roll-up to the executive level? Are there key mash-ups I should/could use?
JET: without digging into specifics with regard to mashups in our Rally tool, I highly recommend looking at a set of metrics we gathered in our whitepaper: “A CIO Playbook for Adopting for Adopting Scrum” co-authored with Ken Schwaber and Dean Leffingwell.
One appendix in the paper guides teams in a quick self-assessment of their Agile uptake. The second offers metrics both teams and executives can use as new measures of success.
Altering how we measure and guide success in Agile has a great impact on the “stickiness” and intended benefits of the Agile rollout. Rally tool mashups simply support real-time access to such measures. This “reporting” style deviates from the traditional “reporting up”. Rather, the metrics we recommend are meant to keep the entire organization informed about what is really happening. This creates organizational ownership of amplified learning and continuous improvement.
Have you integrated user centered design (usability practices) into the Agile development process? If yes, what was your approach? If no, why not?
JET: I believe user-centered design has a place in Agile development. Design emergence doesn’t mean it can only occur in the planning meeting for the next iteration, IMHO.
creates a high-level vision (informed by many sources);
collaborates with the product owners about how to incorporate this across a product roadmap;
provides more detailed guidance about backlog items specific to the user-centered design;
then engages actively within Scrum teams when product backlog items specifically impact user design changes (where she is tester or owner or delivery team depending on the item.)
Do Agile practices only apply to software delivery, or have you ever seen them applied to successful service delivery?
JET: Jeff Sutherland tells a great story about helping his wife apply Scrum in her life as a Unitarian minister. I also have a colleague who uses it in his volunteer organization. And there are people in my Certified ScrumMaster classes from non-software parts of software organizations eager to bring Agile in. Here at Rally, we use Scrum in EVERY department in the company, including at the executive level.
It is already challenging to obtain the time of critical resources (shared among various projects and sustaining). Are every day SCRUM meetings practically possible?
JET: Everyday meetings are faster than having to write them extensive documentation. And, they are less costly than not having them involved at all. Still, teams make agreements about how critical it is for “part-time” experts to be engaged in every meeting. They inspect and adapt.
Is Scrum mostly for R&D projects with more unknowns or is it equally effective on projects with extensive functional and technical knowledge and skills.
In other words, is Waterfall totally obsolete?
JET: Rigid methodologies don’t serve environments where any amount of change can occur. If you have projects that have no change in scope, no mis-interpretations of requirements, no defects in design, no defects found in code test or integration that would require re-work, I see no reason for you to change. I do not know of such projects.
At Rally, we often get questions about how Agile fits in with a variety of other practices and processes, so the below questions about Agile and CMMI, Agile and RUP, and Agile and Architecture come as no surprise. Stay tuned for the final Q&A on Agile champions.
DW – is Dave West from Forrester
RAM – is Ryan Martens from Rally
How can Agile be integrated in a CMMI certified application in regards to the tremendous amounts of required CMMI artifacts (documentation)?
DW – The flippant answer is it can’t. ☺ But actually I have seen some great systems integrators use CMMI to help check list their processes of which some of those processes are Agile. Agile development works really well at the team level, but it needs to be surrounded with other processes. CMMI makes sure you remember those processes, some of which are inside the Agile team, others are not. The PA’s always seem to imply lots of documentation, but they do not have. You can keep the documentation to a minimum, but still deliver them. Do not, however, burden the Agile team with proving their compliance – Use the checklist, build the software and get others to prove they did what they said they did.
RAM – I am not quite as negative as Dave on this topic. CMMI and Agile combine just fine. There are a number of proof points that are easily found from past Agile conferences and by searching “Agile and CMMI.” The problem is that many folk misinterpret CMMI and are convinced that it mandates waterfall, heavy process and heavy documentation. According to SEI, it does not. Most teams would not adopt CMMI without a specific business need.
We received lots of great questions and shared them with the webinar attendees, but thought they would also be valuable to other Agile practitioners. This first series of questions tackle the obstacles of adopting Agile. Stay tuned for the remaining questions in later posts.
What are the biggest blockers you see to good Agile?
DW – My pet peeves are people that resist change without really thinking about it. That treat software like car production and think that you can plan, deliver in that manner. These same people often lament the ability of the software teams to deliver good code at the right price. Software is a fundamental part of business change – If you need a new part to your business it will most likely require software, which may require development and integration. Connecting up their development with the people who have the problem is the first step to success, empowering teams to try new things and question the status quo in support of those business needs is key. Organizations that put artificial walls between the problem and the solution are the biggest blockers. These may come in the form of planning processes, business access, travel budgets, governance models, standards or culture.
RAM – Good Agile development is a mindset change like Dave is describing above. Overly aggressive or passive plans are the biggest blockers to “good Agile.” You need to think about this change as incremental and iterative with a strong focus on continuous improvement and learning through inspect and adapt feedback loops.
RallyDev.com
I give guidance that a shorter timebox is preferable to a longer one. This gives the team that moment of pause for inspection and adaption sooner rather than later.
