The Agile Bet
And now some testimony from Brother Nicely-Nicely Johnson, I mean, James Turner, from O’Reilly Radar:
The Cult of Scrum:
If Agile is the teachings of Jesus, Scrum is every abuse ever perpetrated in his name. In many ways, Scrum as practiced in most companies today is the antithesis of Agile, a heavy, dogmatic methodology that blindly follows a checklist of “best practices” that some consultant convinced the management to follow.Endless retrospectives and sprint planning sessions don’t mean squat if the stakeholders never attend them, and too many allegedly Agile projects end up looking a lot like Waterfall projects in the end. If companies won’t really buy into the idea that you can’t control all three variables at once, calling your process Agile won’t do anything but drive your engineers nuts.
If there’s one thing I’ve learned from my several attempts to implement Agile teams inside large, non-agile corporations it’s that you can have total developer buy-in, test-driven development, daily standups, continuous integration, story points, and iterations, and still not have anything that feels like a functioning Agile project.
Look at it this way. There are two fundamental ways to deal with change on a software project:
-
You can try, up front, to perfectly analyze the project such that there will be no need to change requirements during development. This approach leads to to various artifacts associated with waterfall processes, such as design documents, UML diagrams, and the like.
-
You can try to set up a structure that assumes that change will happen, and will allow you to lower the impact of changes when they come. This approach leads to Agile practices such as short iterations, TDD, and continuous integration.
Agile, then, is a bet. A bet that you can skip a substantial percentage of the time being spent trying to pre-analyze and start coding much sooner in the process. The bet says that if you use the right coding practices and the right process management, your project will deliver more value more rapidly and more sustainably than it would have with the pre-analysis approach. That’s what “responding to change over following a plan” means in the agile manifesto.
At first glance, though, the two options don’t look mutually exclusive. And up front analysis is so seductive. Sure, we can do all our great agile stuff, but what’s the harm in also writing a UML diagram up front? We can be test-driven, but why not also have a rigorous acceptance process? The more stakeholders there are, and the further they are from the development team, the more pressure there is to write things down, get buy-in up front, and generally increase the ceremony around change management. Somebody on every team always thinks they can eliminate future change by getting the right plan up front. Sometimes it’s me.
And look, it might work for you. Every project is different. But there are at least some issues to look out for when you mix waterfall and agile processes together.
You’re hedging your bet. In essence, an Agile project spreads it’s design time across the length of the project rather than having most of it up front. Agile, by design, accepts what might be somewhat slower initial development practices (pairing, TDD), because they lead to a lower cost of change over time. Again, having some design can help, but there comes a point where adding back in the time-intensive design tasks gets you the worst of both worlds – the initial development costs of Agile and the high cost of change associated with a waterfall project.
Worse, the existence of a lot of documentation surrounding changes tends to make the agile project management less effective. The more documentation and the more people that have to sign off on change requests, the less flexible the project is. The less flexible the project, the harder it is to realign priorities in response to development and business realities as they change. It’s hard to reorganize priorities in a planning game meeting if three levels of management have to sign off on scope changes.