March 8, 2020
Now that we know how unpredictable predictions can be, it’s worth asking two questions. Firstly, if the experts are so wrong, why do we make predictions in the first place? And, how do we circumvent following the wrong prediction?
Firstly, it’s important to note that experts are the worst at admitting when they’ve been wrong. According to Tetlock, experts often justify their mis-predictions by insisting they were ‘just off’ on timing or blindsided by an improbable event. The truth of the matter is that even the experts fall victim to the effects of confirmation bias and hindsight bias.
Secondly, the future is non-linear, and that the past is a very poor guide to the future - a sentiment the experts are still catching up on! As tempting as it is to think that in a world ruled by computers and data science, and algorithms we must therefore be able to predict the future with at least a small amount of accuracy, we can’t! We live in a quantum universe where Chaos Theory makes a mockery of every hard-and fast rule based on past performance.
You might know the Chaos Theory in relation to the Butterfly effect - the notion that the flapping of a butterfly’s wings in the Amazon could affect the weather in China. What Chaos Theory does really well is distinguish between two kinds of uncertainty. On the one hand there are some things we don’t know that are, in theory, knowable. For instance, you possibly don’t know how much the building you work in is worth, but someone does; and with enough time and determination, you could find out. That kind of uncertainty is known as epistemic uncertainty. But the uncertainty we’re particularly concerned with, and the kind that drives us towards making predictions, is aleatory uncertainty; essentially that which we don’t know and is unknowable. For example, elections. We don’t know who will win until they do, and every prediction we make up until that point is simply a thinly veiled guess.
Unfortunately, and this is where the experts come in, humans are not very comfortable with this kind of uncertainty, which is why experts appear to be so reliable. We are suckers for what is called the cardinal bias, the tendency to place more weight on what can be counted on than what cannot be. We crave precise answers over vague ones, even if they turn out to be wrong. So, when we want to know something that we cannot possibly have the answers to, we turn to those who say they do, even though they’re most likely going to be wrong.
This level of comfortability essentially answers the question as to why we make or listen to predictions when they are likely to be wrong, which leads us to the second question, how do we circumvent these predictions in order to still be able to look ahead for our businesses. Fortunately, according to Tetlock, we still need the experts. According to his research, not all experts are created equal, or in other words, while many could be replaced by a chimp with a dart board not all of them could be.
So how do we know whom to trust and whom to ignore? To answer this question, Tetlock borrowed from Isaiah Berlin (who in turn borrowed from the Greek poet Archilochus) and divided his experts into two groups - the foxes, and the hedgehogs. In this division, the foxes know many things, while the hedgehogs know one big thing. Tetlock found that the foxes repeatedly outperformed the hedgehogs. Put simply, experts who were comfortable with nuance and contradiction were more likely to produce correct predictions (the foxes), while those who were wedded to just one organising principle were unsurprisingly more likely to be wrong, and, scarily, more confident in their initial predictions.
And the lesson - certitude, is not a measure of certainty! Find a fox, not a hedgehog, to give you witness insight. In other words, you need thinking partners who are as nimble and agile as you’re going to need to be to thrive.
What does this look like then you might ask? At Research First we’d give you a one-word answer. Foresight. Getting hung up on the details of specific predictions is not going to stand you, or your business in good stead, instead you must work to understand the fundamentals that drive the thing you’re interested in. Specific predictions might indeed be a fool’s errand, foresight is not.
To quote Winston Churchill, “It is always wise to look ahead, but difficult to look further than you can see.” Had he been in marketing he might have said, if you have to plan for a future farther than what you can see, plan for a surprise!