The article was written by Henn Ruukel, Principal Group Engineering Manager at Microsoft.
I have seen people and organizations falling into pitfalls using Agile methodologies for their software development – the result is disappointing. I would argue that some of these struggles can be avoided by looking into what Agile methods pledge and what they are not trying to deliver. Having a clear understanding of the aim helps us understand whether they fit our purpose or we should use something else instead.
The central promise of Agile is to help solve business problems as quickly and iteratively as possible so that a company can adapt to changing markets or validate new opportunities. This vow sounds great, but without exploring the other side of the coin, it leaves the expectation that this can be achieved without any trade-offs. Let’s look into some of the most common pitfalls and misunderstandings.
Estimates vs commitments
The first highly stressful pitfall is outcome predictability – to be more precise, the lack of predictability. By nature, Agile methodologies are used in an environment where there is an understanding of the business problem but not how to solve it or the required timeframe.
Initially, it sounds unacceptable for a business to rely on a process where the team working on a task is not able to commit to delivery, almost as if the team is untrustworthy. Especially, if the task is vital for the business.
Think about it as if you were the first person to summit Mount Everest. You have the general skills and experience to climb mountains and a clear understanding of the expected outcome, but little to no knowledge of what it will take to complete the climb or when exactly you’d be summiting.
Many problems businesses face have lots of similarities with this example. They have to accept from the get-go that people working on solving them don’t know how much energy or time it will take. They can provide some estimates, but the only thing a team can commit to is using as small iterations as possible to validate progress and guide the direction of the next steps.
Often, Agile is best suited for these kinds of business problems. They are high risk, but also high reward if solved successfully. In this case, a commitment to the delivery date is less important than the solution itself.
If a business problem has solid external commitments that are costly to miss, one can still use Agile, but additional measures for risk management are essential. For example, a common measure would be active scope management to meet those deadlines.
Discovery vs implementation
The second pitfall for Agile development is mixing up discovery with implementation. Treating discovery and implementation as separate activities does not mean switching back to the Waterfall method. Instead, it means that the mission team is mindful of whether a particular activity is done to discover new information or act on the information we know.
I’ll bring another example from the mountaineering world. The team faces a cliff and they don’t know what’s behind it. They have two options:
Discovery – a fraction of the team will drop most of their gear and do a quick climb over the cliff;
Implementation – the whole crew will climb the cliff with all the gear.
In both cases the team gains new knowledge – the difference is that in the first case, the learning probably happens faster and with less energy spent.
We should apply the same thinking process in Agile software development. We need to use small discoveries, either building prototypes or launching the products to a subset of users. I have found that teams are reluctant to use Discoveries as it feels like throwing-away work. It depends on our certainty of what is behind the cliff.
Engineering efficiency vs business value
The third pitfall I have seen is teams optimizing engineering efficiency over business value. This presumption sounds confusing - aren’t they related to one another? Don’t we want to use our engineering resources efficiently? In order to explain what I mean, it’s probably best to explain different flavors of “efficiency” separately:
Keeping everyone occupied
Although it feels right to optimize teamwork to have enough work for everyone – either in a sprint or in a day – sometimes, this is not an efficient solution. There are many situations where the team moves faster when some members don’t do anything but wait for their turn and preserve energy. Referring to the cliff discovery example, when a few members are on the discovery climb, others should rest to be ready to move forward when the discovery is completed. Same in software development – your non-critical code changes might block releasing critical changes needed for discovery. Not always, but sometimes the whole team would have been able to move forward faster if you had taken a day off.
Avoiding throw-away code
We all would like only to create lasting value in the ideal world. Same with software. It feels unnatural to build something as a temporary and throw-away solution. The challenge here is that the Discovery phase often requires building makeshift solutions. The idea is to validate a hypothesis or increase confidence in a specific approach. One way to think about it is that we are creating throw-away code, but this doesn’t equal throw-away work. The objective is to learn, not to write code.
Specialization
We need to find the right balance between programming tools and within our team set-up. The team has to be best suited and people relatively autonomous and self-reliant during the mission. These two traits combined with tasks where the solution remains unknown indicate that we should optimize for generalists and adaptive people in the team over specialists that are good in a narrow area. The more generic our team skillset is, the less efficient they are at any specific task/area. But we win in the agility to adapt to the environment around us.
Although tempted to optimize for engineering “efficiency” we should zoom out and optimize our decisions against business value creation as fast as possible.
Summary
I hope these pitfalls shed some light on the considerations around Agile. May my thoughts help you navigate your software development process, whether you are a developer, product manager, or a stakeholder of the business.
Agile methodologies have their significant strengths. We must agree on the context and objectives while using it – are we on a mission to climb Everest for the first time, or are we completing a well-defined task in a predictable environment? The answer will tell us whether these strengths outweigh and justify its inefficiencies in some of its aspects.