Now that we've talked about how we got into the bad habits of waterfall processes, let's switch gears and talk about why Agile methods should be embraced within the organization.
Software is expensive. It is a significant investment, and, as with any investment, there is an expectation of a return. In the IT world, the expectation for the return is based on reduced business costs, improved business function, or some other measurable benefit. In the product world, the expectation is that the return will, in some way, generate additional revenue. In any case, if the return is less than the return on a savings account, then the company is better off banking the money.
Am I arguing to not create software? Definitely not. However, I am arguing that thought needs to happen on how to maximize the return. I'll discuss three aspects of that: the costs of quality, the problems of over-engineering, and the pain of failure.
The costs of fixing an error in a software project increases non-linearly over time. These are real costs in terms of resolution time and schedule slippage. The worst case is an error found by a customer, which costs the most, affects new product development, and impacts brand reputation.
Waterfall methods have quality scheduled in, but is not a core to the methodology. Worse, testing is back-loaded on the project, significantly increasing the cost involved to fix a defect.
Agile methods, on the other hand, have a very strong quality focus and testing is front-loaded on the project. By front-loading testing, defects are easier to find and easier to fix, with significantly less cost.
Studies have shown that nearly half of features developed are never used. Another fifth of the features are seldom used. Each of these features increases the cost and risk of the product without providing any additional value.
Waterfall methods try to pull together all the requirements up front. Because of the challenge of prioritizing all of the requirements, I've often seen prioritization happening at the feature or use-case level. This means that if a use case has three scenarios, A, B, and C, and only A is high priority, all three scenarios will be implemented, even though B and C are not important.
Agile methods encourage building only what you know you need as you know you need it. Rather than implement A, B, and C, Agile methods encourage working with the customer at the time of implementation to determine that you only need to build A. B and C are not implemented, so the costs are not incurred from a development and testing perspective.
Failure encompasses two problems: outright project death and projects that become significantly challenged through cost and schedule overrun, poor quality, or developed software not meeting the customer's needs. As software complexity increases, likelihood of failure also increases, reaching as high as three-fourths of projects failing.
As complexity has significant correlation with failure, anything that can be done to reduce the complexity also reduced likelihood of failure. This is one of the great benefits of Agile methods.
I am reminded of the epigram (which is way overused), "How do you eat an elephant?" Answer, "One bite at a time." Agile methods help all aspects of the project to be carved up, from requirements through testing.
This covers the software cost issues at a very high level. In my next installment, I will talk about the suitability of a software project to a given need, problems waterfall methods introduce with that, and how Agile methods shine in solving that problem.