How to make better predictions

Over the weekend I argued that people are really quite good at making predictions, when you zoom out and think of all the various ways we do so in science and in everyday life. Talk about how “predictions are hard, especially about the future” tends to concentrate on a narrow band of particularly difficult topics.

But even in those cases there are ways to improve your ability to predict the future. The classic book on the subject is Phil Tetlock’s Expert Political Judgment which I recommend. And if you want the short version, and happen to have a subscription to The Financial Times you’re in luck: Tim Harford’s latest column there gives a useful summary of Tetlock’s research.

His early research basically uncovered the role of personality in forecasting accuracy. More open-minded thinkers — prone to caution and the appreciation of uncertainty, who tended to weigh multiple mental models about how the world work against each other — make more accurate predictions than other people. (They still fail to do better than even simple algorithms.)

I’ll excerpt just the last bit, which focuses on Tetlock’s latest project, an ongoing forecasting tournament (I’m participating in the current round; it’s a lot of fun and quite difficult). Here’s the nickel summary of how to be a better forecaster, beyond cultivating open-mindedness:

How to be a superforecaster

Some participants in the Good Judgment Project were given advice on how to transform their knowledge about the world into a probabilistic forecast – and this training, while brief, led to a sharp improvement in forecasting performance.

The advice, a few pages in total, was summarised with the acronym CHAMP:

● Comparisons are important: use relevant comparisons as a starting point;

● Historical trends can help: look at history unless you have a strong reason to expect change;

● Average opinions: experts disagree, so find out what they think and pick a midpoint;

● Mathematical models: when model-based predictions are available, you should take them into account;

● Predictable biases exist and can be allowed for. Don’t let your hopes influence your forecasts, for example; don’t stubbornly cling to old forecasts in the face of news.

Humans are great at prediction

Humans are terrible at making forecasts, we’re often told. Here’s one recent example at Bloomberg View:

I don’t mean to pick on either of those folks; you can randomly name any 10 strategists, forecasters, pundits and commentators and the vast majority of their predictions will be wrong. Not just a little wrong, but wildly, hilariously off.

The author is talking specifically about the economy, and I mostly agree with what I think he’s trying to say. But I’m tired of this framing:

Every now and again, it is worth reminding ourselves just how terrible humans are at predicting what will happen in markets and/or the economy.

Humans are amazing at predicting the future, and yes that includes what will happen in the economy. It’s just that when we sit down to talk about forecasting, for some reason we decide to throw out all the good predictions, and focus on the stuff that’s just hard enough to be beyond our reach.

There are two main avenues through which this happens. The first is that we idolize precision, and ignore the fact that applying a probability distribution to a range of possibilities is a type of prediction. So the piece above is right that it’s incredibly difficult for an economist to predict exactly the number of jobs that will be added in a given month. But experts can assign probabilities to different outcomes. They can say with a high confidence, for example, that the unemployment rate for August will be somewhere between say 5.5% and 6.5%.

You might think that’s not very impressive. But it’s a prediction, and a useful one. The knowledge that the unemployment rate is unlikely to spike over any given month allows businesses to feel confident in making investments, and workers to feel confident making purchases. I’m not saying we’re perfect at this probabilistic approach — recessions still surprise us. But it’s a legitimate form of prediction at which we do far better than random.

That example leads me to the second way in which we ignore good predictions. Talk of how terrible we are at forecasting ignores the “easy” cases. Will the sun rise tomorrow? Will Google still be profitable in a week? Will the price of milk triple over the next 1o days? We can answer these questions fairly easily, with high confidence. Yes, they seem easy. But they seem easy precisely because human knowledge and the scientific process have been so successfully incorporated into modern life.

And there are plenty of other predictions between these easy cases and the toughest ones that get thrown around. If you invest in the stock market for the long-term, you’re likely to make money. Somewhere around a third of venture-backed startups won’t make it to their 10th birthday. A few years down the line, today’s college graduates will have higher wages on average than their peers without a degree. None of these things are certain. But we can assign probabilities to them that would exceed that of a dart-throwing chimp. Perhaps you’re not impressed, but to me this is the foundation of modern society.

None of this is to say we shouldn’t hold pundits and experts more accountable for their bad predictions, or that we shouldn’t work to improve our predictions where possible. (And research suggests such improvement is often possible.)

But let’s not lose sight of all the ways in which we excel at prediction. Forecasting is a really broad category of thinking that is at the center of modern science. And compared to our ancestors, we’re pretty good at it.



Does social media make people less happy?

I’ve written a bunch over the last few years attacking the idea that social media is isolating, causes loneliness, etc. But Technology Review has a piece up on some new and seemingly credible research on the relationship between social media use and reported happiness:

But there is growing evidence that the impact of online social networks is not all good or even benign. A number of studies have begun found evidence that online networks can have significant detrimental effects. This question is hotly debated, often with conflicting results and usually using limited varieties of subjects, such as undergraduate students…

…They found for example that face-to-face interactions and the trust people place in one another are strongly correlated with well-being in a positive way. In other words, if you tend to trust people and have lots of face-to-face interactions, you will probably assess your well-being more highly.

But of course interactions on online social networks are not face-to-face and this may impact the trust you have in people online. It is this loss of trust that can then affect subjective well-being rather than the online interaction itself.

Perhaps, like so many other things, the truth is that it depends. Use the internet to meet and stay in touch with a wide circle of people with whom you sometimes interact in person, and it probably makes you better off. Use it to monitor a wide circle of weak ties or strangers with whom you seldom have much interaction, maybe it doesn’t.

I’ve been critical of even that line of thinking, because from the studies I’ve seen previously it’s seemed that online networking has been, on net, positive. So it should be noted that this research isn’t a confirmation of what we already knew. It runs contrary to it. Previous research has tended to find that online activity has been highly social, and hasn’t caused social isolation. (I’m not aware of anything specifically looking at the impact of social media on trust before this.)

No doubt we’ll continue to get a better picture of the interaction between social media and happiness over time. But for now, this research offers a notion of caution for the optimists.