For example, in the post on Poor correlation among critics' quality scores, I compared the quality scores from five commentators over 60 vintages of the Penfolds Grange wine. There was no time trend in the scores for four of the commentators, but the Wine Spectator showed a very clear upwards trend in the scores through the vintages, as shown in the first graph. [Note: most of these wine scores were not given at the time of release, but are based on subsequent retrospective tastings.]
For comparison, the time correlation value for the other commentators ranges from 7% to 18%, versus 58% for the Spectator. So, for the Wine Spectator more than a half of the variation in the scores is associated with time, which is not true for the other commentators.
The line in the above graph is a running average (of 9 vintages), showing that the scores rise until the early 1990s, and then remain somewhat steady after that. Indeed, it was in 1995 that the Wine Spectator named the 1990 Penfolds Grange as its wine of the year.
This intrigued me, so I looked for other long-term data from the Wine Spectator. I looked for a broad range of wine types (different styles from different regions), since the Spectator has a range of different reviewers, and I wanted to include as many of these as possible. For each wine, I wanted at least 10 scores from the period 1975-2014 (40 vintages), with some of the scores before and some after the 2000 vintage. What I came up with is shown in the table, with the time correlation indicated for each wine.
|Schloss Vollrads Riesling Spätlese, Rhine
Ruffino Riserva Ducale, Chianti
Schloss Johannisberg Riesling Spätlese, Rhine
E. Guigal Château d'Ampuis, Rhône
Fontanafredda Serralunga d'Alba, Barolo
J.J. Prüm Wehlener Sonnenuhr Riesling Auslese, Mosel
Viña Tondonia Reserva, Rioja
Penfolds Grange Bin 95, Barossa
Louis Latour Corton Grand Cru, Bourgogne
Viña Tondonia Gran Reserva, Rioja
Château Latour, Bordeaux
Moet & Chandon Brut, Champagne
Château Lafite-Rothschild, Bordeaux
Henschke Hill of Grace, Barossa
Château Climens, Bordeaux
Robert Mondavi Winery Cabernet Sauvignon, Napa
Château Mouton-Rothschild, Bordeaux
Château Margaux, Bordeaux
Stag's Leap Wine Cellars Cabernet Sauvignon, Napa
Joseph Drouhin Charmes-Chambertin Grand Cru, Bourgogne
As you can see, the majority of the correlations are high — in this context, any correlation greater than about 20% is unusually high. As an illustration of what a 25% correlation looks like, in the next graph I have included the data for two of the Bordeaux chateaux for the period since 1980.
It turns out that I am not the first person to have noticed this pattern. In a blog post entitled Fun with wine numbers, Tom Wark looked at the Wine Spectator's point scores for several hundred California chardonnays over 18 years. He summarized the data in terms of the percentage of wines with particular scores; and I have graphed his results in the next figure.
The decrease in the percentage of wines with scores <80 is the result of an editorial decision to stop publishing such scores (there are plenty of wines with high scores to write about). However, the increase in the percentage of wines with scores >90 is precisely what I have shown above for individual wines.
Why has this happened?
This leads inevitably to a consideration of what is causing this time pattern. Tom Wark commented: "I honestly don't know what to make of this. It looks like point inflation taking place ... On the other hand, it just may be that CA Chardonnays got a heck of a lot better."
The Wine Spectator itself agrees with the second option. Writing in that magazine, James Laube (Wine rating inflation) noted: "It's indisputable that wines are better now than a generation ago. Vineyard management, winery technology, winemaker skill — all have progressed. And as wines have improved, ratings as reflected by scores have risen."
However, not everyone else agrees with this idea. If we take the Henschke Hill of Grace wine listed above, for which the Spectator time correlation is 26%, the time correlations for the same period from some other commentators are: Jancis Robinson 0%, Jeremy Oliver 4%, James Suckling 5%, Huon Hooke 25%, and Robert Parker 26%. So, two of the critics agree with the Spectator about increasing quality, and three don't.
It is inevitable that Parker's wine scores went up, of course. Lots of winemakers started making "Parker wines" precisely for the purpose of getting high "Parker scores", and so it is inevitable that Parker would prefer later wines to the earlier ones — they were made especially for him. In this sense, Parker is simply the victim of his own excess — his strong wine bias has helped create a wine world that suits him well, but not necessarily anyone else, including other wine commentators. The wines styles do, however, seem to suit the palates at the Wine Spectator.
This is not really score inflation, but is instead simply another example of confirming the consequent. It is a feedback loop, in which high scores encourage wine makers to produce wine styles that will generate more high scores.
To examine the idea that the Wine Spectator's scores reflect score inflation, instead of better wines, we would need data that are independent of Robert Parker, which we do not have. However, it is worth noting that the Spectator's higher scores for recent wines occur even in vertical tastings, where all of the vintages are tasted at the same time. So, the high scores do not represent a slow creep upwards through time, but are instead a clear preference for modern wines compared to older styles.
Modern wine styles are associated with the change to riper vintages since 1990 (often attributed to global warming; see Fifty years of Bordeaux vintages), which produce "bigger" wines with higher alcohol contents. If you want to read about such things, then Dan Berger has a long series of posts in his USA-syndicated wine column (thanks to Bob Henry for pointing these out to me):