For example, in the post on Poor correlation among critics' quality scores, I compared the quality scores from five commentators over 60 vintages of the Penfolds Grange wine. There was no time trend in the scores for four of the commentators, but the Wine Spectator showed a very clear upwards trend in the scores through the vintages, as shown in the first graph. [Note: most of these wine scores were not given at the time of release, but are based on subsequent retrospective tastings.]
For comparison, the time correlation value for the other commentators ranges from 7% to 18%, versus 58% for the Spectator. So, for the Wine Spectator more than a half of the variation in the scores is associated with time, which is not true for the other commentators.
The line in the above graph is a running average (of 9 vintages), showing that the scores rise until the early 1990s, and then remain somewhat steady after that. Indeed, it was in 1995 that the Wine Spectator named the 1990 Penfolds Grange as its wine of the year.
This intrigued me, so I looked for other long-term data from the Wine Spectator. I looked for a broad range of wine types (different styles from different regions), since the Spectator has a range of different reviewers, and I wanted to include as many of these as possible. For each wine, I wanted at least 10 scores from the period 1975-2014 (40 vintages), with some of the scores before and some after the 2000 vintage. What I came up with is shown in the table, with the time correlation indicated for each wine.
| Schloss Vollrads Riesling Spätlese, Rhine Ruffino Riserva Ducale, Chianti Schloss Johannisberg Riesling Spätlese, Rhine Penfolds Grange Bin 95, Barossa E. Guigal Château d'Ampuis, Rhône Fontanafredda Serralunga d'Alba, Barolo J.J. Prüm Wehlener Sonnenuhr Riesling Auslese, Mosel Viña Tondonia Reserva, Rioja Louis Latour Corton Grand Cru, Bourgogne Viña Tondonia Gran Reserva, Rioja Château Latour, Bordeaux Moet & Chandon Brut, Champagne Château Lafite-Rothschild, Bordeaux Henschke Hill of Grace, Barossa Château Climens, Bordeaux Robert Mondavi Winery Cabernet Sauvignon, Napa Château Mouton-Rothschild, Bordeaux Château Margaux, Bordeaux Stag's Leap Wine Cellars Cabernet Sauvignon, Napa Joseph Drouhin Charmes-Chambertin Grand Cru, Bourgogne |
83% 67% 63% 58% 57% 56% 54% 52% 38% 32% 30% 29% 26% 26% 22% 11% 8% 5% 2% 1% |
As you can see, the majority of the correlations are high — in this context, any correlation greater than about 20% is unusually high. As an illustration of what a 25% correlation looks like, in the next graph I have included the data for two of the Bordeaux chateaux for the period since 1980.
It turns out that I am not the first person to have noticed this pattern. In a blog post entitled Fun with wine numbers, Tom Wark looked at the Wine Spectator's point scores for several hundred California chardonnays over 18 years. He summarized the data in terms of the percentage of wines with particular scores; and I have graphed his results in the next figure.
The decrease in the percentage of wines with scores <80 is the result of an editorial decision to stop publishing such scores (there are plenty of wines with high scores to write about). However, the increase in the percentage of wines with scores >90 is precisely what I have shown above for individual wines.
Why has this happened?
This leads inevitably to a consideration of what is causing this time pattern. Tom Wark commented: "I honestly don't know what to make of this. It looks like point inflation taking place ... On the other hand, it just may be that CA Chardonnays got a heck of a lot better."
The Wine Spectator itself agrees with the second option. Writing in that magazine, James Laube (Wine rating inflation) noted: "It's indisputable that wines are better now than a generation ago. Vineyard management, winery technology, winemaker skill — all have progressed. And as wines have improved, ratings as reflected by scores have risen."
However, not everyone else agrees with this idea. If we take the Henschke Hill of Grace wine listed above, for which the Spectator time correlation is 26%, the time correlations for the same period from some other commentators are: Jancis Robinson 0%, Jeremy Oliver 4%, James Suckling 5%, Huon Hooke 25%, and Robert Parker 26%. So, two of the critics agree with the Spectator about increasing quality, and three don't.
It is inevitable that Parker's wine scores went up, of course. Lots of winemakers started making "Parker wines" precisely for the purpose of getting high "Parker scores", and so it is inevitable that Parker would prefer later wines to the earlier ones — they were made especially for him. In this sense, Parker is simply the victim of his own excess — his strong wine bias has helped create a wine world that suits him well, but not necessarily anyone else, including other wine commentators. The wines styles do, however, seem to suit the palates at the Wine Spectator.
This is not really score inflation, but is instead simply another example of confirming the consequent. It is a feedback loop, in which high scores encourage wine makers to produce wine styles that will generate more high scores.
To examine the idea that the Wine Spectator's scores reflect score inflation, instead of better wines, we would need data that are independent of Robert Parker, which we do not have. However, it is worth noting that the Spectator's higher scores for recent wines occur even in vertical tastings, where all of the vintages are tasted at the same time. So, the high scores do not represent a slow creep upwards through time, but are instead a clear preference for modern wines compared to older styles.
Modern wine styles are associated with the change to riper vintages since 1990 (often attributed to global warming; see Fifty years of Bordeaux vintages), which produce "bigger" wines with higher alcohol contents. If you want to read about such things, then Dan Berger has a long series of posts in his USA-syndicated wine column (thanks to Bob Henry for pointing these out to me):
Have you considered that Spectator uses benchmarks for all their blind tastings? Instead of consistency as a justification for benchmarking it is actually driving scores up on the wines that are benchmarked.
ReplyDeleteYes, that is an interesting suggestion. This would create a feedback loop, leading to a time-course pattern.
DeleteYou write:
ReplyDelete"It is inevitable that Parker's wine scores went up, of course. Lots of winemakers started making 'Parker wines' precisely for the purpose of getting high 'Parker scores', and so it is inevitable that Parker would prefer later wines to the earlier ones — they were made especially for him. In this sense, Parker is simply the victim of his own excess — his strong wine bias has helped create a wine world that suits him well, but not necessarily anyone else, including other wine commentators. The wines styles do, however, seem to suit the palates at the Wine Spectator."
Let me quote Jancis Robinson from her blog titled "Is California dreaming?":
"The main wine-tasting was a fascinating four-hour marathon over a long lunch at Waddesdon Manor, Lord Rothschild's wine-minded estate near Aylesbury. About 20 of us, mainly the American visitors plus myself, Hugh Johnson and Michael Broadbent from Britain and Hardy Rodenstock and Hans Johannson from Germany gathered round a long table in Waddesdon's palatial 'Dairy'.
"We tasted flights of four to six vintages from the nineties of seven of California's best Cabernets old and new: Araujo 'Eisele,' Caymus 'Special Select,' Dalla Valle 'Maya,' Harlan 'Estate,' Ridge 'Monte Bello,' Screaming Eagle and Stag's Leap Wine Cellars 'Cask 23' . . .
"The wines were served strictly alphabetically . . .
"But at this point [after serving the Dalle Valle and Harlan wines] . . . Hugh Johnson declared, 'I find no grace in these wines at all. They're designed for cigar smokers. There's mass but no aroma.' Tough talk.
"The next flight brought some respite for California though in the form of six vintages of Ridge 'Monte Bello,' from 1990 to 1995, the only wine in the tasting from outside the Napa Valley . . . This lovely and well-proven wine was certainly distinctive, consistently delicious and characterful, well balanced, not too big, and the best wine was, tellingly, the oldest. This is a Californian wine made to a more recognisably European formula, one that requires age of a wine . . .
"By this stage a certain gloom had descended on the long table at which were expertly served this succession of glasses (34 apiece for the main tasting alone) and dishes. Had all these dollars really been spent on wines incapable of winning friends with some of Europe's most experienced palates?
"The sea change among the sea of glasses came when the Screaming Eagles were served. European noses twitched appreciatively. Eyebrows were raised and the odd smile played on European lips. I thought they were delicious. . . . Even Messrs Johnson and Broadbent were impressed. . . .
"Tom Black, wine collecting owner of a bank services software company in Nashville, had donated the great majority of these wines. If it had been me I would have felt distinctly protective towards them but he is obviously made of sterner, more objective stuff. 'In California this is what they like. We don't have to like it. We must understand it and recognize the shift in people's taste. I will never believe any of the cult wines in 20 years can compete with a Latour, Cheval Blanc, Pétrus or Haut Brion.'"
For those readers who wish to explore more of California wine writer Dan Berger's columns in the Napa Valley Register newspaper, here's a link to his archive:
ReplyDeletehttp://napavalleyregister.com/wine/columnists/dan-berger/