Forecasting, what is it good for?
History is full of terrible predictions. Some are quite fun in hindsight, such as Ken Olsen’s prediction that:
“There is no reason anyone would want a computer in their home.”
Writing this on my laptop at home sort of feels like I'm personally spiting Ken with each keystroke.
John Maynard Keynes, often considered the father of macroeconomics, made a similarly poor (if not slightly more tentative) prediction that within 100 years:
“… the economic problem may be solved …” (Keynes, J.M., 1930, pg.6)
Such that to satisfy all our needs we may only need to work:
“Three-hour shifts or a fifteen-hour week …” (Keynes, J.M., 1930, pg.8)
Who would have thought Keynes would be such a fan of Degrowth?!
Despite these unfortunately poor predictions, forecasting has actually improved significantly in recent decades. Most of this is due to both an increase in the quality and availability of data, as well as increased computing power to estimate more complex models. For example, Meteorologists have significantly increased the accuracy of weather forecasts since the 1970s.
This is good. However, it hasn’t solved the problem of forecasting and how they’re used.
When predictions go wrong
Quite famously in Australia Philip Lowe, the then governor or the Reserve Bank of Australia (RBA), made a quite general forecast on interest rates:
“This is the basis for our assessment that the cash rate is very likely to remain at its current level until at least 2024". (RBA, 2021)
Now at the time, this was probably not an entirely unreasonable forecast. The pandemic was raging and uncertainty about the economy was high. Through their communication, central banks have power to reinforce confidence in markets. One way to signal this confidence is via lower interest rates. However, Lowe’s forecast obviously didn’t turn out to be a particularly good prediction. In fact, it likely cost them their job.
It’s actually quite an interesting exercise to compare the RBA’s historical forecasts to what actually happened. Below we can see the plots for key macroeconomic variables showing what actually happened (black line) as well as all the historical RBA forecasts (coloured lines) back to the Nov-2018 forecasts published in the Statement on Monetary Policy archives.
We can see that the forecasts are a bit all over the place. Sometimes they follow the trends / direction of what actually happened but are a bit off (especially since ~ 2023). Sometimes, like during the recession of 2020 up to roughly 2023, the forecasts were entirely wrong.
One way to quantify the “error” of these forecasts is through evaluation metrics such as Mean Absolute Error (MAE) or Root Mean Squared Error (RMSE). These metrics indicate the degree to which the predicted value is different to the actual value. So, a higher value means the prediction is further away from the true value, and vice versa a smaller value means it’s closer to the true value.
The graphs above plot these metrics for each of the key forecasted variables. As you can see, if there wasn’t a recession recently, the forecasts actually perform relatively well (are quite close to zero).
This raises an interesting question, why can’t Macroeconomists predict a recession? Shouldn’t Macroeconomics, the school of thought born almost entirely out of the Great Depression, be able to forecast when a recession is going to happen?
Why can’t we predict recessions?
A key issue is likely the over reliance on outdated concepts like the NAIRU and simplified macroeconomic models. For example, the modern canonical Dynamic Stochastic General Equilibrium (DSGE) Models have a lot of very simplifying assumptions.
These assumptions can limit the flexibility of a DSGE model when making predictions. For example, this class of models tend to push an economy towards it’s “steady-state” path. Because the base equations in these models tend to be smooth continuous functions, this makes predicting large discontinuous events, like recessions, quite difficult.
Get Pedagogy of the Economists’s stories in your inbox
Join Medium for free to get updates from this writer.
Other approaches that relax some of these simplifications are being explored, such as “agent based modelling” (Janssen, M.A. & Ostrom, E., 2006). In theory, ABM’s might be able to better fit the discontinuous nature of many recessions. Recent research has built an ABM that competes with standard forecasting methods when predicting the COVID-19 recession in Austria (Poledna, S., et al., 2023).
Further to ABM approaches, economists using different methods of analysis did identify the potential risks of high debt levels to the financial sector prior to 2008 (Rajan, R.G., 2006). Similarly, accounting based models helped predict debt issues in the financial sector (Bezemer, D., 2009). Obviously, these forecasts were not given the same weight as the existing orthodox economic models at the time.
What is it good for?
“All models are wrong, but some are useful.”
How useful are these macroeconomic models that cannot predict major economic crises? Sure, you could argue that there is pedagogical value in abstracting the truly colossal complexity of an entire economy into a “representative agent”. However, this simplification means we miss out on important qualitative differences.
If the models predictions only consider the “representative agent” of an economy, we entirely miss out on the importance of distribution. Does this representative agent account for the diversity of our society? Can it account for how different people respond to price and non-price signals differently? Unlikely.
One alternative is to employ a pluralist approach through forecasting competitions. This involves asking forecasters to offer both their predictions, and their confidence in those predictions. These competitions also seeks to reward behavior that leads to better forecasts. A pluralist approach that adopts multiple different forecasting methodologies could help remove the reliance on a single model, approach, school of thought or dataset. This has the potential to paint a more holistic picture of what the future holds and the confidence in these potential futures.
This is all good and well, but we should remember that forecasts are just that, forecasts. The uncertainty principle indicates that it is truly impossible to know what will happen for certain, until it happens. The more you know about the momentum of a particle, the less you know about its position, and vice versa. That is until you observe it. Similarly, the more we know about the economy at time t, does not necessarily tell us more out the economy at time t + 1. You can never know both for certain, prior to observing them both.
Thus, whilst forecasts can provide important information, we should not be placing too much weight on forecasts alone. Other qualitative information is essential. For example, when setting interest rates, considering the impact that interest rates have on the distribution of income, might be important to consider.
To the degree that forecasts help improve the participatory planning of the economy, they have value. Just as weather forecasts help people plan for when to take a raincoat, or when to prepare for a disaster, economic forecasting can play a similar role. In other scenarios the normative values that underpin our society should play a larger role in informing planning decisions.
“What do we want our society to look like”? Whilst forecasts can help inform an answer to a question like this; foresight, consensus building and a participatory economic democracy are the keys to both providing and shaping a good answer to this question.
“We ought, for example, to be thinking far more in terms of the issue of foresight rather than an ideological precision in forecasting.
Foresight and forecasting are not the same things.
Forecasting is about stating the future that will be, foresight is about anticipating possible futures and shaping them — perhaps avoiding what may otherwise be, and where one considers a possible state of affairs to be adverse.” (Morgan, J., 2012)
As we saw from the poor predictions at the start of this essay, a nice prediction is useless unless it comes true. Too increase the likelihood of a forecast you want to come true, you should be asking not what the forecast can do for you, but what you can you do to help make a better society.
The future is green from the grassroots. Only we can plant trees today, that we know we will never sit in the shade of. No forecast can do this for us.
References
Bezemer, D., 2009. No one saw this coming. Understanding financial crisis through accounting models.
Morgan, J., 2012. Forecasting, prediction and precision: a commentary. Economic Thought, 1(2, 2012).
