Monday, February 4, 2019

How many 100-point scores do critics really give?

There has been much talk about the apparently inexorable increase in the values of wine-quality scores over the past few decades. The scores from 1983 (when Robert Parker introduced his 100-point scale) to the year 2000 averaged much less than have the scores from 2000 to 2018. Indeed, Jamie Goode noted two years ago: "The 100-point scale is very compressed at the top end …. and this scale is becoming so bunched at the top end that it is nearing the end of its useful life."

This leads to the obvious question about just how crowded the top end might be, irrespective of whether we use 100 points or 20 points, or something else.


Before looking at the actual data, though, it is important to note that there are two possible interpretations of a maximum-point score: (i) the wine is as good as we expect to meet in our lifetime; or (ii) it is the best that could ever be. If we mean the latter, then we run the risk of claiming that we are the arbiters of perfection. As Ambrose Bierce defined it:
PERFECTION, n. An imaginary state of quality distinguished from the actual by an element known as excellence; an attribute of the critic.
So, it may be best to claim that we mean option (i), not option (ii). Under these circumstances, of course, when we do subsequently do encounter a better wine then we would need to assign a score in excess of 100 points (see Why not expand the 100-point scale?).

Some data

One suitable place to look for data about how often wine commentators use maximum scores is the Wine-Searcher database. Here, we are provided with hundreds of thousands of wine-quality scores from 30 critic sources, or so (some of which represent groups of people). For most of the critics we can, at the click of a button or two, get a list of their 500 top scores. This allows us to compile the data reported here (mostly compiled at the beginning of this year).

The only tricky data to compile come from Robert Parker himself, or more generally his publication the Wine Advocate, as the scores actually come from a number of people. The issue is that there are more than 500 100-point scores in the database. For example, Lisa Perrotti-Brown recently noted just how many 100-point scores the Advocate has for the 2016 Napa wines, alone. So, I would like to thank the people at Wine-Searcher (especially Robert Anding) for looking up some of the numbers for me.

The data that I compiled cover 23 critics who use the 100-point scale and 6 critics who use the 20-point scale. In each case, I calculated the percentage of their scores that are the maximum; and for the 100-point scale also how many were near-maximum (99 and 98 points).

Proportion of maximum wine-quality scores from selected commentators

The first graph shows the 29 critics, ranked in order of how many of their scored wines received maximum points. Not unexpectedly, Robert Parker and the Wine Advocate team are the principal culprits, although Jeff Leve (at the Wine Cellar Insider) is trying very hard. However, quite a range of the other commentators have non-negligible numbers of top scores. Indeed, only 9 of the critics (one-third) have no maximum-point wines in the database. Interestingly, this includes 3 of the 4 critics from Australia (Jeremy Oliver, Huon Hooke, and the three-headed Wine Front).

This leads me to wonder whether some of these people are giving high scores but without being willing to produce the attention-getting maximum scores. I examined this by looking at the percentage of scores covering the 98-100 points range for those 23 critics using the 100-point scale. (This does not really work for the 20-point scales, since the data collection would involve half-points, which Wine-Searcher does not record.)

Proportion of 98-100-point wines from selected commentators

These data are shown in the second graph, with the critics still listed in the same order as above. This shows that everyone uses near-top scores, and that some people use them a lot. In particular, Luca Gardini has no 100-point scores but quite a few at 98 and 99 points. Furthermore, Tim Atkin and Daniele Cernilli (at Doctor Wine) clearly use an over-abundance of 98 and 99 points, compared to 100 points.

If we exclude the Parker/Advocate scores, then 0.3% of the scores in the database have maximum points (ie. 3 out of every 1000 wines); and 0.8% of the scores are in the range 98-100 points. These numbers may be lower than many people are expecting. This seems to be mainly because 34% of the Wine-Searcher scores actually come from the Wine Spectator magazine (or 32% if we include all of the database scores), and its contributors produce relatively few scores in the 98-100 range (0.07%). The Wine Enthusiast is the next-biggest contributor (17% of the scores), and even it has only 0.16% of its scores in the 98-100 range.

Finally, it is instructive to look at the four Australian critics plus the lone New Zealander. James Halliday has long been singled out in Australia for handing out a lot of high scores (eg. What's in a number? Part the second), and the data show that he does indeed use 100 points more than do any of his compatriots. However, the 98-100-point data show a very different picture. Jeremy Oliver is the only one with fewer 98-100-point wines than Halliday; and Huon Hooke and Bob Campbell exceed Halliday to the tune of 3.0 and 3.6 times as many wines! Apparently, only Oliver has not yet succumbed to the lure of the high-level scores.

2 comments:

  1. For the longest time, the only wine that Stephen Tanzer had awarded a "100-point" score to was . . .

    2010 Egon Muller Scharzhofberger Riesling Trockenbeerenauslese

    . . . which sent the price ZOOMING in the "after-market" of fine wine stores that had any bottles for purchase.

    Today on Wine-Searcher it costs U.S. $8,681 for a 375 ML bottle, on U.S. $10,563 for a 750 ML.

    URL: https://www.wine-searcher.com/find/2010+egon+muller+scharzhofberger+riesling+trockenbeerenauslese/1/united+states

    More recently, Tanzer added a few other wines to his exalted 100-point ranking:

    2009 Chateau Margaux

    2012 Domaine de la Romanee-Conti Romanee-Conti Grand Cru

    2009 Chateau Yquem Sauternes

    1999 E. Guigal Cote Rotie La Ladonne

    1998 Domaine Zind-Humbrecht Pinot Gris Rangen de Thann Sant Urbain Selection de Grains Nobles

    1998 Domaine Zind Humbrecht Riesling Rangen de Thann Clos Saint Urbain Selection de Grains Nobles

    ("Bragging rights": I was recently able to "score" 375 ML bottles of the 1998 Domaine Zind-Humbrecht Pinot Gris Rangen de Thann Sant Urbain Selection de Grains Nobles here in the States for less than U.S. $100 a bottle. No, that's not a typo. A great example of what I call "wine arbitrage.")

    ReplyDelete
  2. EXCERPTS FROM THE DRINKS BUSINESS
    (May 7, 2015):

    "[ROBERT] PARKER [SAYS]: NOT AWARDING 100 POINTS 'IRRESPONSIBLE';
    Wine critics who fail to give perfect scores are 'dodging responsibility' according to the world’s most influential wine reviewer, Robert Parker."

    URL: https://www.thedrinksbusiness.com/2015/05/parker-not-awarding-100-points-irresponsible/

    By Patrick Schmitt

    "During an interview with the drinks business earlier this year, Parker – who developed the 100-point rating system – expressed his urge to award full marks to great wines, and his dismay at those who don’t.

    "'When, in your mind, the wine is the best example you have ever tasted of this particular wine, you have an obligation to give it a perfect score,' he told db.

    "On the other hand, he branded those who are incapable of awarding a perfect score 'irresponsible'.

    "'I think the person who can’t give 100 is really dodging responsibility, because there’s no way they haven’t tasted a wine that is the best example they have tasted from this producer, the best example they could ever think of.”'

    "He then stated, 'I think it’s irresponsible not to give a perfect score if you think the wine is perfect.'"

    FOLLOWED BY THIS OBSERVATION:

    "Parker also admitted that he himself has been disappointed by his own perfect scoring wines.

    "'How often do I go back and re-taste a wine that I gave 100 points and repeat the score? Probably about 50% of the time, but most of the time – and there have been a few exceptions -- I can understand why I did see it as perfect at that time.'

    "Finally, Parker said it was 'the emotions of the moment' that made the difference between a very high scoring wine and one that’s worthy of the full 100 points.

    "'To me the difference is the emotions of the moment -- the wine must evoke emotion -- just like art or music or beauty, there should be an emotional response, and great wines should be emotional.'

    "Continuing he stressed, 'It is that little extra dimension of emotion that comes with 100 points, and that’s the difference between a 97, 98, or 99 and 100.'"

    AND THAT "EMOTION" IS WHAT MAKES AWARDING WINE RATINGS LESS-THAN-METHODICAL, AND SUBJECT TO WHIM AND CAPRICE.

    ~~ BOB

    ReplyDelete