RallyDev.com
Tue 26 Jan 2010
You Have To Buy It Twice Before It’s Cheap
One of the most common sources of tension between product owners and developers is when product owners are surprised at how high an estimate for a story might be. Usually this tension is easy to resolve by reiterating that the product owners really have no concept of how much something should cost. However, there is one scenario I see over and over again: when a product owner protests the estimate of a story because it seems, to the PO, like it’s simply re-using an aspect of the system somewhere else.
“I already paid for that!” he or she might say, quite correctly. A simple example recently for us here was our column picker. Our grids allow you to select which columns you wish to see on any given grid. Later on, we created the ability to have grids inside of panels on user dashboards. Our product owner wanted to put the column picker on the settings pane of a grid panel. To him, this was simply reusing the column picker that had already been created for an earlier story. He was quite shocked when our estimate for the effort was just as high as it was the first time.
His frustration is completely understandable. How is this possible? Why isn’t it easy to just drop that component into another place and have it work? After all, if we were good engineers, wouldn’t we have written that component to be reusable?
Actually, no. It’s because we were good engineers that it wasn’t written in that way. This is one of the hardest things for product owners to understand, and I hope to explain it in this post.
Evolutionary Design
Once upon a time, development teams worked in specific phases. First they would design all of the code they were going to write; boxes would be drawn on whiteboards, sequences captured in documents, and so on. Once the design was finished, the developers would go off and write the code for it. Luckily, our industry has realized that this is a problematic way to write software.
It’s simply not likely that the team will discover all of the quirks of the design during the design phase. It’s even more likely that requirements and features will change during the long span of time this process takes, forcing the developers to throw away work and go “back to the drawing board” on the design. Splitting the design phase from the coding phase is a recipe for disaster.
Instead, it’s generally preferable to practice evolutionary design; allow the design to grow organically as the code is written. This, combined with testing and refactoring, seems to be the best known way to ensure that the quality of the code remains high as a product grows.
Doing this makes it easier to deal with changes that effect design, and it prevents the team from wasting time going “back to the drawing board” in the event a problem is discovered.
YAGNI
One principle to follow as you code is the YAGNI principle. YAGNI means “You Ain’t Gonna Need It” and essentially it means that developers should discourage themselves from adding additional code to support something that is not, at the time, required. Developers have a strong desire to add code here or there while editing code, and YAGNI helps us remember to resist the urge to add superfluous code or functionality.
Following this principle helps eliminate waste, and helps keep developers focused on adding value that product owners want. But why?
If I’m editing Module A to add Functionality X, why not just add Functionality Y while I’m there? It may delay the delivering of Functionality X in the short-term, but later when it’s decided that Functionality Y must be supported as well, it will be dirt cheap, right?
The trick to understanding why YAGNI helps is realizing that there are thousands upon thousands of potential Module As in a system. The truth is, Functionality Y may simply never be required. And even if it is, Functionality Z may not be. Adding that functionality would be a waste of time and resources if it’s never used.
YAGNI Applied to Design
The same principle behind YAGNI can be extended to aspects of high-level design. When we originally designed the Column Picker component, it was not designed to be dropped into any part of the system. This was intentional: if we had spent the time to make the Column Picker completely generic and reusable, but then had never needed to drop it into another part of the system, the additional effort required to make it super-generic would have been a wasted effort. This is true for any aspect of the system being designed: we can spend the extra time and effort to make something extremely generic, but if that generality is never needed, that time is waste. Many times developers will refer to something that’s overly generic as “over-engineered.”
This means that the description of the component can often be misleading. We never built “a column picker” – we built “a column picker for a grid page”. The column picker was written in a way that assumed it was part of a grid page. Again, it could have been written more generically, but doing so would have potentially created waste (this does not mean that it’s alright to design code that violates principles of good design. Code can be well-designed but still not fully generic).
So when it came time to put the component in a completely different context than the one for which it was designed, it was still a lot of work: almost as much as was originally required to make the first pass. Though some time was saved simply because it was a known and well-understood problem for which we would leverage newly-developed knowledge, it still effectively required writing a whole new component: one that was much more generic and utilized what it could of the previous increment.
Essentially, to avoid having developers waste time, components should be “generic on demand”. Once a component needs to be reused, then it should be refactored into something extremely generic, but not before. The end result is that a product owner will have to pay for a component twice before it’s cheap.
This may seem counterintuitive to product owners, but it’s important to understand. Because the team approaches coding in a way that causes the second time to be nearly as expensive as the first, the team is able to deliver everything else quickly.
The first time, it is costly because the developers have to solve unknowns and create something new from scratch.
The second time, it is costly because it has to be molded into something generic and reusable.
The third time, however, is cheap.
Great post. I totally agree with the YAGNI as I spent 3 years on a frameworks group where we built full generic componetry that eventually got used in a single place wasting all the effort of making it reusable.
YAGNI seems like a politically safe way of development while shifting and increasing costs later in the project. If a developer writes a lot of extra code that isn’t immediately used it looks like they are trying to run up the meter. But if he or she writes barely enough code to make it work and has to completely rewrite later, they appear to be simply trying to keep costs down. Reuse of work is economical in software and unlike in other professions there is cost savings in building a repertoire of generic pieces from the start. There just has to be enough experience to do so successfully.
YAGNI is not useful for those writing the 50th column picker of their career or for those who would later have to deal with their own refactoring debt. Wasted effort becomes acceptably small when developing something that requires much, much less effort due to experience. Developers who own (in a literal sense) the code they write also gain advantages in putting extra effort behind the code. Product owners can’t capitalize on reusability and code qualities that they have no way of becoming aware of, so the up front effort would be wasted in the more typical employee’s situation. Add to that the fact that the project can be pulled away from a developer literally at any time, and it’s too risky sometimes for developers to pay in up front.
Given enough time, non-developers with a short term stake in a project will YAGNI it to death, preventing both forward planning and refactoring and then move on once the project becomes unmanageable and the proverbial soil becomes infertile.
For those interested, there is some further discussion going on around this post on Reddit – http://www.reddit.com/r/programming/comments/aulpi/you_have_to_buy_it_twice_before_its_cheap/
Dallas:
If you’re writing the 50th column picker of your career, then You Are Gonna Need It.
Like I say in the article, I’m not advocating writing crappy code. Code can be well designed without being infinitely reusable in any context. It’s because the code is good that the second story costs about the same as the first, rather than 100 times more.
The column picker we built WAS reusable – as long as it was kept in the same high-level context. Once the context changed, it wasn’t reusable enough.
My point in the article is that a component can always be MORE generic until it’s completely, 100% generic and reusable in all contexts (almost a library at that point). But if you approach every part of the system with the goal of making it that generic, you are likely to be generating a great deal of waste, since the majority of components won’t actually be reused.
Once something has proven that it WILL be reused, then it should be made into something extremely generic. Again, at no point should the thing ever be poorly written or poorly designed.
There’s a reason why you haven’t taken your company’s application and rewritten it so that it’s contained entirely in configuration files that get read into a super-generic framework you’ve written. It’s because you’re a good engineer, so you are following this principle, whether you like it or not.
I cannot agree more.
What you’ve done is highlight the fact that you didn’t actually agree upon the same thing; there are hidden assumptions in your customer’s mind and in your developers’ minds that were not properly called out, discussed, costed and agreed upon.
It doesn’t matter whether you do Big Design Upfront or evolutionary design, if your customer thinks one thing and you develop another thing, then there will be disappointment on both sides.
If you had asked them about reusability, your customer may well have requested you to develop it the first time so they could reuse it the second time, despite your developer’s YAGNI principles. It’s just another case of following the customer’s expectations rather than developers playing a guessing game.
I agree with what you said Rod. Re-factoring and making things extremely generic can go too far. There has to be a balance.
Its like the people who love to create tons of interfaces and abstract classes just in case…
Sometimes its just not worth all that extra code.
I do agree with this article, and wish more companies and developers understood this principle. It has been stated that generic software (truly generic, and not just a stab) cost 3-5 times that of specific software. So, if you are going to reuse the software more than three times, then fine, write is generic from the beginning. Once the “reuse” issue arises, then handle that problem by writing a more generic piece of code. NOT BEFORE. Solving problems before they exist is always tempting to the beginner or intermediate programmer is tempting. The more experienced programmer will wait, then, with more information, the generic code will be better than the earlier generic code just by reason of more information.
Wait until you need to use the code in more than two places before you write generic code. Once you have three examples, then you will have a much better idea how to solve the general problem.
The comment Jonathan Chauncey mentions is exactly right. Don’t make a Rube Goldberg solution just because it uses every feature you just read about in the new “Useless Patterns for Everyday Programming.”
[...] You Have To Buy It Twice Before It’s CheapThe trade-off between re-usability and YAGNIagile software reuse YAGNI [...]
Genericity should indeed begin at the second time. Even if you never need it the third time, you’ve eliminated code duplication, you’ve clearly described the commonality in your program structure.