How We Misinterpret Electoral Polls
Polls are often used as forecasts, when really they should be used as part of a model to forecast.
Thank you for reading our work! Nominal News is an email newsletter that focuses on the application of economic research on current issues. Subscribe for free to stay-up-to-date with Nominal News directly in your inbox:
If you would like to support us further with reaching our 1,000 subscriber goal by year end, please consider sharing and liking this article!
Polls, and especially election polls, are often a lively discussion topic. Over the last several years, many articles and opinion pieces even argued that polling is no longer reliable or wrong. This polling ‘criticism’ was not only confined to the US – other countries, such as the UK, also have had similar opinion pieces. Over this time period, I have also become skeptical of polls. However, thanks to the writings of fellow Substack writer – Carl Allen – it turns out that polls aren’t wrong. It is how we use them that is the issue, and our mis-use of them relates to many of the same issues we have been highlighting at Nominal News regarding economics discourse.
Polls and Forecasts
Most polls that are of interest to the public are about something that will happen in the future. The most common one is elections, but the public is also often polled on various policy questions. In the context of elections, pollsters ask the public which candidate or political party they prefer, and then report the results. A typical outcome could look the following:
Candidate A: 45%
Candidate B: 43%
If the result are presented more thoroughly, we might see one more category:
Undecided: 12%
Based on the above, we would often see headlines that say Candidate A is ‘leading’ or ‘likely to win’. This seems ‘intuitive’. However, what if a poll showed the following results:
Candidate A: 15%
Candidate B: 13%
Undecided: 72%
In this instance, it does not seem that clear to us that Candidate A is ‘leading’ or ‘likely to win’. How come?
Polls aren’t Forecasts
It is natural to assume that a poll result today is a forecast of what will happen come election day. After all, why do a poll if we cannot use it as a forecast? But come election day, when election results are announced, there is generally no such category as ‘undecided’ – people either vote for Candidate A or B, or don’t vote (for simplicity, we’re abstracting from third candidates).
Thus, when saying that the first example poll (Candidate A with 45%) implies that Candidate A is likely to win, we’re implicitly (and potentially unknowingly) making an assumption about what ‘undecideds’ will do. For example, if we assume ‘undecideds’ split 50-50, then Candidate A will win 51% to 49%. But, what if 75% of the ‘undecideds’ are more likely to vote for Candidate B. Then, we could have an opposite result – Candidate B wins with 52%.
If we observe such a result (Candidate B winning), we might look back at the poll that had Candidate A in the ‘lead’ and ‘blame’ the pollsters for being ‘wrong’. But the pollster never made any mistake. The results the pollster presented were correct – it is the assumption we as readers made that was wrong.
In the second polling example, where Candidate A had 15% support and B had 13% support, given the number of ‘undecideds’ that we do not know how will vote, we are more hesitant to say Candidate A will win. We do not feel as comfortable to assume how ‘undecideds’ will vote.
Forecasts are Models
Anything pertaining to the future requires some form of modeling. If we are interested in forecasting elections, we need to ‘model’ how undecideds will vote, which requires assumptions. As we have discussed previously at Nominal News, many economic disagreements are misunderstanding of assumptions. Once the assumption is stated clearly, for example “I believe all undecideds will split more or less evenly’, then we can have a productive discussion about this assumption.
An example of an election forecast model commonly talked about in the US is the “Thirteen Keys to the White House” by Lichtman and Keilis-Borok. Interestingly, this model, which lists 13 yes or no questions, such as whether the economy is in a recession or whether the candidate is an incumbent along with a few subjective questions like the charisma of a candidate, does not take pre-election polls into account at all. This does not mean that this model is necessarily good or bad, but it is a good example of how a model, data and assumptions work together in making a forecast.
In terms of building a forecasting model for elections, one could (and probably should) include poll results, as a data input. Many other factors might also be relevant, such as macroeconomic indicators, demographics, previous elections, and so on. All these factors could be used to build a model that can be tested to see whether the model is a ‘good’ predictor of elections.
When newspaper articles or opinion writers started claiming that ‘something is wrong with polling’, in reality, it was them using a bad model of assuming that a poll is the same as a forecast. The model was wrong, not the poll.
Polling, Opinion and the Lucas Critique
Still, we often, implicitly or explicitly, assume that a poll (or poll margin – the difference between the candidates) is a good forecast for what will happen come election day. One potential issue with such a model, where we assume ‘polls = forecast’ is what is often called in economics the ‘Lucas Critique’. That is, both voters and political candidates may respond during the time period between the poll and election day in a way we did not forecast with our polls. Recently, we had an example just like that – the French parliamentary elections.
Prior to the elections, there were three major parties vying for the top spot. One of the parties was ‘leading’ in the polls and many in the media were talking about the party having an out-right majority in the French parliament. Interestingly, France has two rounds of voting – if no candidate in a district achieves 50% of the local vote, a second round occurs a week later, during which candidates that do not meet a certain threshold, do not participate. This resulted in many districts having elections contested between two or three-way parties. The winner in the second round wins the election. Right after the first round, the two parties in second and third place in the polls cooperated by agreeing to field only one of their candidates in many districts (the candidate that finished third in a district withdrew their candidacy). Had these two parties not done this, in many of the three-way races, the ‘leading’ party would have won, potentially securing a majority in the parliament. However, by withdrawing a candidate, the three-way races became two-way races, in which in many cases the ‘leading’ party lost, as the other candidate had the backing of two parties, resulting in a vote share above 50%. After the second round of election, the party that was leading in the polls and even potentially projected to get more than 50% of the seats in the parliament, finished on a national level in third place!
This above example shows the Lucas Critique in action – the people that were using polls to ‘forecast’ the number of seats the leading party would get did not take into account the fact that the other parties might respond by cooperating. This cooperation completely changed the electoral outcome.
Policy Opinions and Forecasting
Opinion polling, especially on economic issues, is also often used incorrectly as a ‘forecast’. Polls may often ask whether a person supports a particular economic policy or not, if it were to occur. The problem arises that often what people may think at the time of a poll may not be what they think when a policy is enacted.
Recently, at Nominal News, we discussed traffic, congestion and congestion pricing. New York City was in the process of enacting congestion pricing. When congestion pricing was halted, one argument used was that polling showed that people oppose congestion pricing.
In a study by Borjesson, Eliasson, Hugosson and Brundell-Freij (2012), congestion pricing in Stockholm was viewed unfavorably, with only 36% of the public supporting it. However, after a trial period during which congestion pricing was enacted, 53% of people in Stockholm voted in favor of keeping congestion pricing in a referendum. A year later, polling showed congestion pricing was supported by 66%, which further increased several years later to 74%.
This creates a dilemma around certain economic policy decisions. If people are likely to be favorable to a policy after it is enacted, should the policy be enacted even if ‘polls’ currently show that people oppose it. This question is beyond the scope of our work here at Nominal News.
Economics, Forecasts and Elections
At Nominal News, I set out to show how the tools and methods used in economics can be applied to a wide breadth of social topics. Even forecasting elections or what people will think about a policy in the future requires tools and concepts often used in economics like models, assumptions and the Lucas Critique. In election polling, it turns out that many news headlines are not clear about their assumptions (whether consciously or unconsciously), resulting in the incorrect conclusion that something must be wrong with the polls.
If you are interested in learning more about the nuances of polls versus forecasts, I highly recommend the Substack by Carl Allen – Poll Data Series with RealCarlAllen. On this Substack, you can also find forecast models for the US election and also the UK election that recently occurred.
Interesting Reads from the Week
Note: Adding to our previous article on the impacts of heat, higher temperatures reduce learning outcomes: “Without air conditioning, a 1°F hotter school year reduces that year's learning by 1 percent. Hot school days disproportionately impact minority students, accounting for roughly 5 percent of the racial achievement gap.” Thank you to Jadrian Wooten for pointing out the study!
Note: NYC piloted five free bus routes over the last year with very interesting results. Overall ridership on these routes went up by around 30%-38%. Unfortunately, the free fares program will be scrapped because: “Our hope was this pilot would get people out of their cars and onto buses on these routes,” said Demetrius Crichlow, the interim president of MTA New York City Transit. “We did not see anything that aligned with that initial intent.” As we wrote – this result was actually expected. Traffic does not fall because of public transit due to the ‘fundamental law of road congestion’.
Article: Automation and The Limits of Contemporary Economics: Unraveling Economics tackles the issues of Artificial Intelligence and automation and the current approach of economists to the issue.
Article: “What Makes Student Loans So Forgivable?”: Very interesting discussion by B.P.S. on why student loans are different than other loans.
If you enjoyed this article, you may also enjoy the following ones from Nominal News:
Inflation – The Story of 2023 (January 1, 2024) – the biggest economic story of 2023 in the US was the fall in the inflation rate. What caused the fall and why certain measures may have been slightly over-stating inflation.
Monopolies and Antitrust Enforcement (November 20, 2023) – with news of the Federal Trade Commission suing Amazon for anti-competitive practices, economic research suggests that such lawsuits typically improve overall welfare.
Industrial Policy – An Unspoken Issue (August 31, 2023) – many countries are turning towards industrial policy. But few discuss the issue of what will happen to workers once the subsidies to the manufacturing sector end.
Great work, and thanks so much for the shoutout!