Don’t Tie Yourself to a Number
Don’t Tie Yourself to a Number
To draw appropriate conclusions from data, we can’t just assume that if the data is correct, it should also be applied.
Thank you for reading our work! If you haven’t yet subscribed, please subscribe below:
If you would like to support us further with reaching our 1,000 subscriber goal by year end, please consider sharing and liking this article!
As Nominal News grows larger, we will be able to make this a full-time project, and provide more content. If you would like to suggest a topic for us to cover, please leave it in the comment section.
Several of my recent articles have referenced the debates around particular measures (measures of inflation and measures of affordability in the context of housing) and whether the measures are ‘correct’. I do not mean whether measures are incorrectly computed or ‘falsified’, but rather whether the numerical estimate that comes from the measure is capturing the thing it is referencing. Let’s elaborate.
Measurement
A measurement is a well-defined procedure to quantify a characteristic of an object or event. Most measurements are named to reference something we are interested in knowing the number of. For example, when we ask a person’s height, the measurement procedure we think of is using our standardized length system – inches or centimeters – and determining how many inches or centimeters a person is from head to toe. The number established by this process is referred to as height. Note, the measurement procedure can have uncertainty, which we refer to as measurement error. That is, because our measurement tool is imprecise, when someone says their height is 180cm, in reality their height might be between 179cm and 181cm.
Interpreting Measures
We often use measurements to draw a conclusion. For example, when we go to a doctor, the doctor measures your blood pressure. The blood pressure measurement, as in how we do the procedure, is standardized and well established. This blood pressure reading is used as part of the doctor’s analysis in making decisions to improve a patient’s health. If the doctor sees a normal blood pressure reading, then you may be told you’re all fine. Suppose next time, you show up to the doctor dehydrated. Dehydration increases blood pressure, resulting in an elevated reading. The doctor now may look at this reading and state you’re no longer healthy and prescribe medication to lower your blood pressure. But in reality, you are healthy and the cause of the elevated reading was simply a temporary dehydration. Your ‘underlying’ blood pressure, which is what is key in determining your health, was fine.
Economics and Social Questions
The blood pressure example was an intentional oversimplification to illustrate how we can misuse a measurement. We can quite easily see the ‘error’ in this example – the patient does not have an elevated resting blood pressure. In the context of economics and social issues, however, we may encounter similar situations regarding measures, but it is much harder to determine if we are drawing the incorrect conclusion, as opposed to the blood pressure example.
Let’s look at the inflation rate. The reason central banks focus so much on this measure is that it is critical for the healthy functioning of the economy (it is also part of their official mandate). The ‘inflation rate’ is a concept that emerges from a model of the economy. The central bank has a model that simplifies the world, and one of the variables in this model is something we decided to call the ‘inflation rate’.1 This model also has other elements, such as total production (Gross Domestic Product), total capital, and also the main central bank policy tool – interest rates. The model allows us to figure out what will happen to the economy (including the inflation rate) if the central bank changes interest rates.
Naturally, a model requires data. Thus, we need to find out what the ‘inflation rate’ equivalent in our model is in the real world. To do this, we designed measures that aim to capture this model equivalent variable called the ‘inflation rate’. We actually designed a few of them – Consumer Price Index (CPI) and Personal Consumption Expenditure (PCE) – and even further versions of them such as Core CPI, Core PCE, Supercore CPI, etc.
Each of these measures has a specific measurement procedure. None of these measures are wrong. A measure can’t be wrong, in the same way that a blood pressure reading is not wrong. However, since we are using the values of these measures in our model to represent the ‘inflation rate’ variable, we are explicitly assuming this measure reflects this ‘inflation rate’ variable exactly.
This assumption is where all the debates (and many of our articles) have been focusing on. Are the CPI or PCE measures currently reflecting the model-based inflation rate? As we have argued, during typical times (i.e. not during a pandemic), these measures reflect the model-based inflation rate quite well. But due to the pandemic shock and peculiar behavior of housing prices2, along with a unique component in the CPI (Owner’s Equivalent Rent – OER)3, there appears to be a disconnect between these measures and the model-based inflation rate. As previously discussed, I noted that a lot of the elevated inflation observed today (3.2%) is driven not by current inflation, but by rent increases that occurred a year ago.4
PCR Testing Analogy
To give a better analogy for the discussion around inflation and OER, let’s turn to PCR (polymerase chain reaction) Covid Testing, which is the nasal swab test. The PCR test works by taking a sample and amplifying the sample to make additional “copies” of the DNA segments in the sample. The number of DNA matches to the Covid virus is then recorded. If a certain threshold is reached, the sample is described as positive and the person is told that they have Covid.
The amplification process made it slightly more likely to also find ‘dead’ Covid in the nasal swab sample. Thus, even if a person no longer had active Covid, the PCR test measurement could state that the number of DNA matches in the sample is high enough to meet the threshold criteria and result in a false positive diagnosis.
Note – the measurement done by the PCR test is not at fault, as it gives the numerical result it is supposed to. But it is how we used the information that led to a false positive. We concluded that the person has Covid, when actually they had Covid and no longer have it.
Regarding current inflation – it may be similar. The CPI and PCE measures, like the PCR Test, are being computed appropriately. But the conclusion – that inflation is elevated right now (Covid positive) – may be wrong if we are, in fact, capturing the past inflation (dead Covid).
A Measure is Just a Measure
The inflation example shows us the pitfalls of trusting a measure to interpret a model-equivalent concept. There are many other similar cases, where the measure we cite is not actually a good representation of what we are trying to understand. This often leads to fruitless debates.
Another recent example is the ‘recession’ discussion in 2022. To determine whether the US economy is in a recession, many people often point to the quarterly GDP (Gross Domestic Product) growth rates, and informally state that two quarters of falling GDP is a ‘recession’. However, it turns out that the National Bureau of Economic Research (NBER), which announces recessions, is a bit savvier and uses a range of metrics to determine whether we are actually in a recession (based on the strong economic performance following the consecutive negative GDP growth in the first half of 2022, the NBER was right not to label it a recession).
Another example, which we recently discussed at Nominal News, is housing affordability. Again, there is an issue with this term because everyone uses a different measure to get at the concept of affordability. Commonly, individuals reference house prices or rents as a fraction of income. But as we have discussed, transportation is also a critical factor influencing people’s ability to afford to live in an area. Maybe a more representative measure of what we mean by ‘affordability’ is income over housing plus transportation cost.
There are also even broader conceptual issues. For example, we look at GDP and wage growth and state that this means the economy is doing well. But what if this growth is due to people working longer hours – is that a good or bad thing? To answer these types of fundamental questions, it's often not enough to look at one particular measure.
Summary
You may have heard the phrase that ‘you can prove anything with statistics’ or economics for that matter. I used to agree with this statement, but have since changed my mind on it. The issue with the statement is that it undermines the fact that there is genuine work done to create and improve measures, as well as incorporate more measures that allow us to draw good conclusions.
Of course, it is easy to create a statistic or a measure and claim that it implies a specific preferred conclusion. Upon closer inspection, however, it would be straightforward to determine that the underlying assumptions of such a statistic (or measure) are not actually a good reflection of what we are claiming to try to understand. When we ‘lie’ with statistics (or measures), we are simply obfuscating these assumptions, hoping no one notices.
Interesting Reads from the Week
News: The Federal Reserve kept interest rates steady. Based on the forecasts of the Federal Reserve voting members, three interest rate cuts are expected in 2024.
- talks about the recent changes in realtor fees in the US and how this change might play out in the market.
- discusses personality traits and, fittingly to the article discussed today, what are the way personality traits have been measured (along with a link to the “Big Five” personality test).
Photo by Ono Kosuki.
If you enjoyed this article, you may also enjoy the following ones from Nominal News:
Inflation – The Story of 2023 (January 1, 2024) – the biggest economic story of 2023 in the US was the fall in the inflation rate. What caused the fall and why certain measures may have been slightly over-stating inflation.
How the Internet is Changing Economics (December 3, 2023) – the Internet has had a meaningful impact on certain economic relationships that used to hold true in the past. The creation of new markets and products used to be considered a positive, but findings regarding social media suggest otherwise. Separately, online shopping may have made us less price-sensitive, impacting inflation dynamics.
Surveys, Assumptions and Understanding Housing Supply (October 14, 2023) – a Financial Times opinion piece discusses research claiming people do not understand the economics of housing supply. Under closer scrutiny, it is not as clear cut as it may seem.
The inflation rate is the change in the price level in the model, where the ‘price’ is a single variable that is a weighted average of all the prices of goods in the model economy.
House prices and rents spiked significantly in 2022, growing by over 15%, and then gradually dropped to a 3% growth rate in 2023/2024.
OER is a measure of how much a homeowner would hypothetically be paying in rent. This homeowner is not actually paying this.
Separately, if we were to use the Euro-zone wide inflation measurement method – the Harmonized Index of Consumer Prices (HICP), which does not include OER – the US inflation rate would be at 2%.
Thanks for the mention! Also, this article was interesting. It’s an important read for many people because they throw these terms around without knowing how they are really measured. Have you read The Deficit Myth? I’d be curious to know your opinion.