The point of this post is that the differences between them are not subtle, at all. Indeed, their differences are important to our success when making decisions about the future, although writers seem to be rather confused about the difference.* They criticize the forecasters because their "predictions" are not accurate — however, the pundits did not make predictions, they made forecasts.
A helpful analogy
For the purposes of this blog post, imagine one of those old-fashioned collections of pigeonholes, in which hotel keys and letters are stored (see above). Furthermore, imagine that each one of the pigeonholes represents a piece of time, and that the events of that time are written inside a single envelope within each pigeonhole. Due to "time's arrow", as the physicists call it, we cannot open these envelopes in any arbitrary order — we must proceed sequentially through the pigeonholes (perhaps starting at the top-left of the collection).
At any given moment, we can read the contents of one pigeonhole only — we call this the "present". We have opened all of the envelopes leading up this one envelope — we call this the "past". We cannot go back and read those envelopes again, but instead we have to remember their contents, perhaps relying on our own memory, or perhaps we wrote a few brief notes at the time. All of the remaining envelopes remain unopened — we call this the "future".**
In spite of our lack of knowledge, we are interested in the contents of the unopened envelopes, because that would tell us what will happen to us next. Perhaps we don't want too many details about some of those envelopes, because one of them tells us when, where and how we will die — that envelope may be best left sealed until we get there!
However, the other envelopes contain information that it would be valuable to know beforehand. For example, in the wine industry they will tell us about future grape-growing seasons, about future wine-making techniques, about future wine-consumption trends, and about future marketing and selling opportunities. A peek into some of those envelopes would make a lot of people rest more easily at night.
But, since we cannot peek, we instead use guesses, predictions or forecasts — we hold the envelopes up to the light, and we try to read what is written inside.
Guesses
Anyone can make guesses. Every time we call heads or tails when flipping a coin we are making a guess. So, we all understand guesses, because we can all do it.
Sometimes we make what we call an "educated guess", which simply means that we have some previous information that might improve the success of our guessing. However, even this does not involve any formal procedure for making the guess — we merely modify our guess in some subjective way, based on what we already know.
There is nothing intrinsically wrong with guessing — after all, we will be right some percentage of the time. Indeed, the old finance "40% Rule" says that one can look good simply by following any proposition with a 40% probability (see How do pundits never get it wrong? Call a 40% chance).
The basic issue, however, is that guesses almost always involve either optimism or pessimism — people guess the outcome that appeals to them at the moment, either because they want some particular thing to happen or they want to stop something. There is too much unhelpful emotion in guessing.
Predictions
Predictions might lie at the other extreme from guesses, in that there is supposedly an explicit method being used to produce the prediction. However, the method of producing the prediction is often not revealed, so that exactly how predictions differ from guesses is not always clear.
A prediction simply declares: "This is what will happen next." This approach leaves the door open for charlatans to make predictions, just as much as it does for people with a serious method of prediction. The world of illusions masquerades as "magic" solely based on this principle. Believing in magic is like believing in Santa Claus — it is fun as a form of entertainment, but it should not be taken too seriously (except by the very young).
This, then, is the basic issue with predictions, that there is no known method of making them repeatable (intuition, crystal balls, tarot cards, consulting prophets, etc). I can predict the weather in any way I like, just as I can predict the outcome of tomorrow's football match, whether or not I tell you how I am doing it.
Forecasts
The difference between prediction and forecasting is pretty simple. Forecasting says: "If things continue the way they have in the past, then this is what will happen next." Prediction leaves out the caveat, and simply declares: "This is what will happen next." For example, we cannot forecast the result of a single toss of a coin, but we can forecast the overall outcome of a long series of tosses; on the other hand, we could predict the result of a single toss of a coin.
So, forecasts involve the relative probabilities of future events, whereas predictions are usually presented as being more certain. Forecasters do not usually eliminate alternative possible futures, they simply claim that they have small probabilities of occurrence. Dealing with a probabilistic world is not always easy for people (see The best way to think about probabilities).
So, technically, "weather forecasting" is not the same as "weather prediction"; and the various weather bureaus around the world insist that what they are doing is forecasting the most likely weather not prediction the future. They do not have a crystal ball, just a bunch of equations. In a previous post, I noted that people are actually smarter than the skeptics think. We keep records (ie. we describe the world), we think about the patterns observed in those records (ie. we explain the world), and we work out how we might respond (ie. we try to forecast the future). So, weather forecasting is not necessarily a futile exercise, as many people have suggested (see my blog post on Foretelling the weather.)
Forecasts often involve some mathematical model — we feed information about the present and the past into the mathematical equation(s), and a forecast comes out the other end. As an example, in my post on Getting the question right, I presented a couple of simple economic models, and used one of them "to explore possible forecasts for future [wine] export growth."
The basic downside to forecasts is that the future is often disconnected from the present and the past. The forecasters have only the present (which we can measure) and the past (which we have recorded) — if either of them turns out to be irrelevant, then the forecasters don't have any more ability than anyone else to talk about the future. There are "unforeseen circumstances" that can arise, and when they do all, forecasts go out the window. For example, tomorrow's weather forecast is unlikely to be correct if a volcano erupts overnight. If you treat a forecast as a prediction, then you are likely to be disappointed in the outcome (for an example, see How the oil rally took forecasters by surprise).
Furthermore, different mathematical models can produce different forecasts, so that there is much gnashing of teeth during any discussions about the appropriate model to use. This is at the heart of many discussions about Global Warming, for instance. To return to my analogy, changing weather patterns are self-evident when we compare today's envelope to the ones we have opened in the past, but it is not always obvious how to use this information to think about the unopened ones.
I noted in a previous post that forecasts are problematic no matter how they are implemented, so that the modern reliance on Artificial Intelligence (AI) will not necessarily help. Nevertheless, the AI forecasts are expected to improve through time, as more and more data are gathered, and the AI system continually adjusts itself based on newly found patterns in the data (ie. they take the newly opened envelopes seriously).
A few wine-related examples
(i)
Consider the following forecasting example, which is of interest to grape growers in California (see Climate scientists see alarming new threat to California).
The most recent drought in California was caused by a high-pressure system in the north Pacific. A low-pressure system circulates air clockwise, while a high-pressure system circulates air anti-clockwise. Therefore, north-Pacific low-pressure systems direct the flow of moist tropical air towards California, while high-pressure systems direct that air toward Alaska, instead. So, with a low-pressure system it rains in California (and there is a drought in Alaska), and with a high-pressure system it rains in Alaska (and there is a drought in California).
For the wine business, then, the important forecast is whether the future holds the prospect of more high-pressure ridges in the northern Pacific. The current forecasting models say that, if the polar ice-cap continues to melt, then: "yes".
Note the three components of the forecast: (i) the "past", which consists of observations of the behavior of high- and low-pressure weather systems; (ii) the "present", which consist of observations of melting ice up north; and (iii) the "if", which relates to whether the past and current situations will continue into the future. Points (i) and (ii) refer to envelopes that we have already opened. Point (iii) is the basic assumption of forecasting that makes it different from a guess or a prediction.
(ii)
Another recent example of forecasting is in the new book edited by Kym Anderson and Vicente Pinilla (2018) Wine Globalization: a New Comparative History (Cambridge University Press). This book (and its companion data volume) provides an extensive coverage of all sorts of wine-related data from 1835 to 2015, intended to illustrate time trends in the wine industry as a whole. This is ideal base material for forecasting, because it quantitatively lists the contents of all previous envelopes, plus the present envelopes. Here, we can genuinely say that if things continue the way they are, then the immediate future can reasonably be forecast; and this is what happens in Chapter 18 of the book (by Kym Anderson and Glyn Wittwer: "Projecting global wine markets to 2025").
Mind you, the book also illustrates the potential folly of trying to extend forecasts too far into the future — after all, even weather bureaus only produce 10-day forecasts. Things do not "continue the same" for long, and the wine industry in every country provides good examples of ups and downs. For instance, the European Union once dominated world wine exports (even as recently as 1988 it accounted for c. 80% of exports), but it no longer does so (<50% in 2014) — if we want to extend this downward trend indefinitely, there must come a time when the EU has no wine exports at all!
(iii)
Finally, a much earlier example of forecasting was the attempt in 1990 by Orley Ashenfelter to forecast the future quality of Bordeaux red-wine vintages, based on observed relationships between vintage quality and summer temperature. In a previous blog post (Fifty years of Bordeaux vintages), I noted that his forecast for the 1989 vintage was spot on, while his forecast for the 1986 vintage was way off. This inconsistency answers the question recently posed by Peter R. Orszag: "An economist’s method predicts a vintage’s quality using only statistics. Why hasn’t it caught on?" More consistency is needed, that's why.
Issues with forecasting
I cannot finish this post without mentioning one further characteristic of forecasts that we all need to be aware of — some events are consistently much easier to forecast that are others. I once wrote a whole blog post about this, using sports forecasting as my example: Forecasting and predicting sports results. [Please ignore the fact that the post claims to have been written by someone named "Christina Pikas" — the file has been corrupted, but that blog is now defunct and I cannot get it fixed.]
The point of the example is that it is easier to forecast sports events that are played until a specified goal is reached, compared to sports events that terminate at a specified time. For example, a tennis match proceeds until someone wins two sets (or three in some competitions), while a basketball game ends at the moment the buzzer is sounded. In the latter case, the game situation even one second before or one second after the buzzer is immaterial, and therefore forecasting the situation precisely at the buzzer is very hard. In one case (tennis) you are forecasting general superiority of the players, and in the other (basketball) you are forecasting a specified moment in history.
Keep this distinction in mind when you are evaluating a forecast — what chance did the forecaster actually have of ever getting it right?
Conclusion
Don't take a guess, and don't make predictions. Forecasts do have their limitations, but they can also do a lot for us.
* The title of the best-selling book by Ian Ayres, Super Crunchers: How Anything Can Be Predicted, is a classic example of confusing the words "prediction" and "forecasting". The book is an interesting discussion of using Big Data for forecasting, but it never once mentions oracles (ie. predictions).
** This idea comes from Fred Hoyle's novel October the First is too Late.
No comments:
Post a Comment