In previous posts, I described how Lean Discovery incorporates concepts from Lean Startup and Lean UX, as well as 4 signs that indicate your organization could benefit from this approach. In this post, I will discuss how even more conservative organizations operating in reasonably well-defined domains could significantly benefit from Lean Discovery as a risk management strategy.
Honestly – when did you last see a risk management plan that addressed concerns like “our customers may not like the product as much as we think” or “the new features could fail to have the business impact that we anticipate?” Arguably the biggest risk to any project is whether it delivers the expected value or simply gets canceled before any significant value is ever delivered. And yet most risk management plans tend to focus on operational and technical issues like the availability of infrastructure resources or retaining the project’s tech lead.
How come?
Well, one reason might be that we generally describe what a project is supposed to deliver in the form of requirements, whether those are in the form of Agile user stories (hopefully!) or long and detailed requirements specification documents (really?). The problem here is that the term “requirements” implies much more certainty and confidence in these specifications than warranted in many cases.
We all know that getting complete, correct, unambiguous and achievable requirements is not an easy task and creating a more voluminous requirements specification document, unfortunately, doesn’t really help. Agile development addresses this problem by putting working software in front of users to get rapid verification that the team’s understanding of the requirements was correct and we “built the thing right.” However, what Agile doesn’t explicitly address is whether the requirements as stated actually represent the most valuable thing we could be developing for our customers based on our current understanding.
To make things worse, it turns out that figuring out “the right thing” is unfortunately much more difficult than we generally assume: studies at very high-performing organizations show that only 10 to 30% of improvement ideas actually generate the expected business value. In other words, even domain experts are wrong about what really provides value to the customer up to 9 times out 10! Even resoundingly successful products were often started based on a completely different value proposition than they ended up with. One somewhat recent example: Slack started out as an internal messaging system during development of a video game that never went anywhere while its erstwhile support tool is now being hailed as the replacement for email.
So, what does this have to do with risk?
Douglas Hubbard, author of “How to Measure Anything,” defines risk as “a state of uncertainty where some of the possibilities involve a loss, catastrophe, or other undesirable outcome.” In other words, uncertainty in our requirements translates directly into the risk to our business value proposition. This is why Lean approaches emphasize validated learning to reduce uncertainty, instead of focusing exclusively on the output of working software. Similarly, the Scaled Agile Framework (SAFe) explicitly considers the “Risk Reduction-Opportunity Enablement Value” (or, slightly less wordy, the “information discovery value”) of a feature when calculating the Cost of Delay. Put even more simply, the countermeasure to feature risk is not additional upfront analysis, it’s knowledge gained from actual feedback. So when phrases like “we’re not sure how popular this feature will be” or “we only have limited data from our user research” dominate the conversation in our early project meetings, then it is generally a good idea to at least initially prioritize development efforts by “information value” rather than by guesstimated financial metrics.
Identifying key assumptions about how our product or feature will achieve the desired business impact allows us to look at them as hypotheses instead of requirements. Jeff Gothelf and Josh Seiden, authors of “Lean UX,” suggest the following hypothesis statement:
We believe that [doing this] for [these people] will achieve [this outcome]. We’ll know this is true when we see [this market feedback*].
If this sounds vaguely familiar, that’s probably because it contains all the elements of a traditional user story, just slightly rephrased to acknowledge uncertainty and replacing acceptance criteria (verification) with measurable success criteria (validation). Obviously, not every requirement needs to be tested with its own experiment. The key is to identify the crucial assumptions we make about how certain features will provide value to our customers and how in turn the resulting behavior changes of our customers will impact our organization’s goals. One great tool to derive features from organizational goals and identify the underlying assumptions at the same time are Gojko Adzic’s Impact Maps.
Surfacing assumptions is risk identification.
Validated learning is active risk mitigation.
Since the purpose of formulating a hypothesis is to test a value proposition without expending a lot of resources (in case it doesn’t hold up), we base our assessment on the customer response to a Minimum Viable Product (MVP), aka the cheapest and fastest thing we can do or build to test whether our assumption is true or not. Most of the time, your MVP doesn’t actually have to be a working product – Dropbox famously tested interest in its envisioned service offerings by creating an explainer video long before they had figured out all of the difficult technical issues of sharing files across a variety of platforms. As a result, their beta waiting list went from 5,000 people to 75,000 people in just a couple of days. How’s that for validated learning? Apparently the makers of Slack are currently checking whether this trick will work similarly well for them…
Once you do need actual functioning code, though, Agile development, and DevOps are essential to allow for rapid and relatively cheap delivery of the working software to put in front of your customers.
In today’s rapidly changing and ever more competitive environment, betting all of your project resources on a single set of value assumption and hoping that they turn out to be true (and still valid) at the end of your project is probably the ultimate risk. If your project has a budget of $10 million and you won’t find out what value the project will really produce until the very end when you can observe the impact of your solution, you carry the entire $10 million as a project risk. Perhaps even more, when you consider the opportunity cost of alternatives that didn’t get build because all the resources were already otherwise allocated.
Shifting our mindset by acknowledging the inherent uncertainty in determining “the right thing” allows us to better manage this risk by:
- surfacing the value assumptions buried in our requirements,
- quickly, cheaply, and systematically validating them with experiments based on real user responses,
- failing fast when we don’t achieve the desired impact, and
- continuously adjusting our roadmap towards maximum value.
The worst kind of risk is the one we aren’t even aware of. Using Lean Discovery to determine and validate the key assumptions about our project’s value proposition allows us to both identify and mitigate the biggest and yet least acknowledged the risk of them all – that we may be building the wrong thing.
* or other measurable impact
