If you’re not failing half the time, you’re not learning fast enough
There are many takes on the “learning from failure”, from the extreme of “celebrate failure” to the opposite of “you only learn from success”.
Like all well-meaning absolutes, some are conflated from personal anecdotes, preferences and confirmation bias. Maybe you prefer not to remember your failures, or need a rational to justify them to upper management. They’re safe to ignore.
The above quote rings true because it emerges from practical experience, but it’s not really meaningful, is it? Should I venture to fail half the time? What would I learn from that? Why would I be better off?
So let’s try to unravel the apparent correlation and see if some prescription emerges.
Let me preface by saying that I’m not talking about developing yet another expense report system. There’s really no excuse to fail on that one. I’m talking about solving a problem that has never been solved before, or attempting to solve a problem in a radically better way.
I’m talking about the process of finding a yet unknown solution to a, sometimes unknown, problem. And it may involve experimenting not just with products and algorithms, but also customer types and markets.
When you don’t know the future, you’ll need to invent it. “Learning” is just another way of saying that. Learning about the problem, learning about the product, learning about the customer.
It turns out that you get the highest information content from the least probable events:
Critical information is usually associated with what a mathematician would call a low-probability event. … the amount of information conveyed by an event is greater the less probable the event. Telling someone in Texas that it’s hot on July 4th conveys a lot less information than if you said it was July 4th and snowing! The latter would be front-page news.
So let’s ignore the stock features that do not differentiate a product, like signing up or using new JavaScript frameworks. What we’re looking for are the unexpected success that comes from experimental features and new ideas.
In fact, we’re not just passively “looking for” anything. We don’t have time for that. We’re actively searching for these unexpected successes. It could be a new algorithm that gives us the edge, a new feature that delights customers, a better viral loop, a new class of unserved customers.
We have a search space of unknown possibilities and we want to minimize the number of steps we need (aka time before we run out of cash) to find those low probability events. Ideally the successful ones, but if we know all the successful outcomes, they wouldn’t be low probability, would they? We have to accept an equal chance of failure.
Our best bet at this point is to repeatedly half the solution space. At each step we’ll either double down on a successful outcome, or back out of a failed path. In short, we’re going to run a binary search.
All things considered, we’re going to be wrong about half the time. But it’s not the act of being wrong half the time that teaches us anything new. Being wrong is easy, just look at news channel pundits. Rather, it’s the process of rapid elimination without bias towards known (and less interesting) outcomes that we end up here:
You learn fastest, while failing about half the time
[1] For the mathematically inclined, I believe the earliest cite of this principle would be Ralph Hartley’s 1928 paper, Transmission of Information.
[2] For more wisdom about software development, I would recommend The Principles of Product Development Flow
Jack Dempsey
Assaf