Why Economists Don't Actually Make Predictions
Don’t ask economists to predict the future. Ask them what the impact will be. It sounds similar, but it isn't.
Nominal News is an economics newsletter written by a PhD Economist that translates the latest economic research into clear, policy‑relevant insights on current issues. Join 4,000 readers to stay-up-to-date with Nominal News:
In the next few months, I will be going full time and turning on a paid feature. If you’d like to help me with some feedback (and a chance to win a 3-Month Paid Subscription tier) – please fill out the questionnaire below:
Nominal News Reader Questionnaire
I recently had a conversation on Substack Notes about economists being bad at predictions. I agreed. But that’s because economists are not in the business of prediction, but something slightly different – causation.
What You’ll Discover Today:
Difference between prediction and causality;
The somewhat unintuitive way economists think (i.e. by holding all else constant);
Why economics is not the best for predicting outcomes.
Prediction vs Causation
New York City recently implemented congestion pricing (a toll to enter the city center). Prior to the launch of congestion pricing, 730,000 vehicles would enter the congestion zone daily. If I were to tell you that economics research tells us that the implementation of the toll would reduce the number of cars entering the zone by 10% – how would the number of vehicles entering the zone change one year after the implementation of the toll:
A: 10% lower
B: the same
C: More than 10% lower
D: Can’t tell
The correct answer, from this list, is D – can’t tell. How come?
A Subtle Nuance
Most economics questions can be boiled down to one formula1:
Y is the outcome variable that we are interested to see how it changes. In our current example, it’s the number of cars entering the congestion zone.
X is the policy variable that we can control. In our case, the toll.
Z is everything else that also impacts the number of cars entering the zone. For example, this can be things like population changes, big events like the world cup/concerts, weather patterns, etc.
Beta is ultimately what economists want to establish – what is the effect of changing policy X on Y. In our hypothetical example, I said that if we enact the $9 congestion toll, the number of cars entering the zone will fall by 10%. That would be the estimate of Beta.
So why can’t we say that the year after congestion pricing is enacted, the number of cars entering the congestion would go down by 10%? That’s because the things that are in Z can change next year, independently of our policy. Thus, the number of cars entering the zone can really take on any value depending on how Z changes.
However, if somehow, nothing in Z changed year to year – Z remained constant – then Y would fall by 10%. This is exactly what economists do say – the $9 toll would reduce congestion by 10%, holding all else constant (i.e. nothing in Z changes).
So what if we implement the congestion toll and next year we observe 600,000 vehicle entries? Since the toll only reduced vehicle entries by 10% of 730,000, the congestion toll only accounts for 73,000 of the 130,000 fall in vehicle entries. That means that something in Z must have also changed that pushed vehicle entries down.
Counterfactual
Another way to think about this is through a counterfactual approach. Had we not implemented the congestion toll, there would have been 73,000 extra vehicle entries. So instead of 600,000 that we actually observed, we would have had 673,000 vehicle entries.
Prediction
People are generally interested in prediction rather than causality. In our case, the news headlines will always simply report the total vehicle entries and that’s what people will focus on. From this number, everyone will erroneously make judgments.
In our little formula above, it means the news and people are really interested in variable Y, and not Beta (the number economists care about).
To predict Y accurately, you not only need to know Beta and X, but also take a stab at forecasting what will happen with all the Z variables. Economists typically do not do that and our tools are generally not good for this. If you want to get predictions of Y, do not ask economists. We can only tell you how changing X will impact Y (the Beta). Our tools are designed to estimate this Beta number well.
Causality
I hope the above explains the nuance between causality and prediction. Causality can inform predictions, but it is not a prediction itself.
I like to think of causality as fundamentally similar to the rules of physics. The causal impact estimated by economists in any context is no different from, say, Newton’s laws of gravity. I do not view the law of gravity as a form of prediction.
The only reason why economists hesitate to call anything ‘laws’ in our field is because establishing laws in economics is much harder than in physics. That’s because we cannot control many things like physicists can – if they want to run an experiment, physicists can often build a physical tool that removes all or most of Z (the large hadron collider in Europe is a prime example of such an item).
Since in economics, we study people, such experiments cannot be run.
Human Nature
The problem economists face is that the economics’ way of thinking is not intuitive. People always want to know what Y will be and not how X impacts Y (Beta). One of the first questions economists get in social settings is always of the type “where do you think the economy is going?”2. I cannot answer that question. I can, however, tell you that if you enact congestion pricing you will have 10% fewer cars, holding all else constant.
People want to know what GDP will be next year, what inflation will be, what their income will be. They directly attribute what they end up observing to the policies that were enacted.
But this is incorrect. Outcomes are impacted by policy, but the specific level of the outcome (actual GDP, inflation, income, etc.) depends on a lot more than just one policy. Often, these non-policies dominate the impact of any policy.3 It still makes the policy worthwhile, but we shouldn’t expect massive changes when things are influenced from a thousand different sources.
P.S. Economists and Predictions
One thing I wanted to add after finishing this piece, is that lately there has been a noticeable increase in economists making predictions, especially around genAI. Recently, 200 economists signed a letter warning about the dangers of AI. It is also not uncommon to find futuristic predictions from economists in social media. I don’t have issues with people discussing the future, but these predictions are personal opinions, rather than causal statements backed by economics research. But, as was the case with the letter, there is significant emphasis on the fact that economists are the ones making these predictions, with the purpose of giving these statements additional validity. This implicitly presumes that economists are better futurists than any other profession, which we are not.
As I laid out in this piece, there’s nothing about economics that really makes us any better at prediction.
Key Takeaways:
When you see a result in economics, remember to interpret it by holding all other variables and policies constant;
Do not decide if a policy worked or not just by looking at the outcome variable — it can deceive you in either direction;
Economics does not have the best tools for making predictions, but it can tell you quite precisely how a policy change will impact the economy.
Please let me know what you thought of the article length:
If you would like to support us in reaching our subscriber goal of 7,000 subscribers, please consider sharing this article and pressing the like❤️ button at top or bottom of this article!
This is my simplification of the Ordinary Least Squares regression model for the purposes of explaining the difference between the Beta parameter and the dependent variable Y.
I was actually asked that question by a border agent once.
A classic example can be house prices. Building additional housing does reduce house prices, but growing incomes increase them. The latter is often far larger than the former effect.