How many 100-point scores do critics really give?

There has been much talk about the apparently inexorable increase in the values of wine-quality scores over the past few decades. The scores from 1983 (when Robert Parker introduced his 100-point scale) to the year 2000 averaged much less than have the scores from 2000 to 2018. Indeed, Jamie Goode noted two years ago: "The 100-point scale is very compressed at the top end …. and this scale is becoming so bunched at the top end that it is nearing the end of its useful life."

This leads to the obvious question about just how crowded the top end might be, irrespective of whether we use 100 points or 20 points, or something else.

Before looking at the actual data, though, it is important to note that there are two possible interpretations of a maximum-point score: (i) the wine is as good as we expect to meet in our lifetime; or (ii) it is the best that could ever be. If we mean the latter, then we run the risk of claiming that we are the arbiters of perfection. As Ambrose Bierce defined it:

PERFECTION, n. An imaginary state of quality distinguished from the actual by an element known as excellence; an attribute of the critic.

So, it may be best to claim that we mean option (i), not option (ii). Under these circumstances, of course, when we do subsequently do encounter a better wine then we would need to assign a score in excess of 100 points (see Why not expand the 100-point scale?).

Some data

One suitable place to look for data about how often wine commentators use maximum scores is the Wine-Searcher database. Here, we are provided with hundreds of thousands of wine-quality scores from 30 critic sources, or so (some of which represent groups of people). For most of the critics we can, at the click of a button or two, get a list of their 500 top scores. This allows us to compile the data reported here (mostly compiled at the beginning of this year).

The only tricky data to compile come from Robert Parker himself, or more generally his publication the Wine Advocate, as the scores actually come from a number of people. The issue is that there are more than 500 100-point scores in the database. For example, Lisa Perrotti-Brown recently noted just how many 100-point scores the Advocate has for the 2016 Napa wines, alone. So, I would like to thank the people at Wine-Searcher (especially Robert Anding) for looking up some of the numbers for me.

The data that I compiled cover 23 critics who use the 100-point scale and 6 critics who use the 20-point scale. In each case, I calculated the percentage of their scores that are the maximum; and for the 100-point scale also how many were near-maximum (99 and 98 points).

The first graph shows the 29 critics, ranked in order of how many of their scored wines received maximum points. Not unexpectedly, Robert Parker and the Wine Advocate team are the principal culprits, although Jeff Leve (at the Wine Cellar Insider) is trying very hard. However, quite a range of the other commentators have non-negligible numbers of top scores. Indeed, only 9 of the critics (one-third) have no maximum-point wines in the database. Interestingly, this includes 3 of the 4 critics from Australia (Jeremy Oliver, Huon Hooke, and the three-headed Wine Front).

This leads me to wonder whether some of these people are giving high scores but without being willing to produce the attention-getting maximum scores. I examined this by looking at the percentage of scores covering the 98-100 points range for those 23 critics using the 100-point scale. (This does not really work for the 20-point scales, since the data collection would involve half-points, which Wine-Searcher does not record.)

These data are shown in the second graph, with the critics still listed in the same order as above. This shows that everyone uses near-top scores, and that some people use them a lot. In particular, Luca Gardini has no 100-point scores but quite a few at 98 and 99 points. Furthermore, Tim Atkin and Daniele Cernilli (at Doctor Wine) clearly use an over-abundance of 98 and 99 points, compared to 100 points.

If we exclude the Parker/Advocate scores, then 0.3% of the scores in the database have maximum points (ie. 3 out of every 1000 wines); and 0.8% of the scores are in the range 98-100 points. These numbers may be lower than many people are expecting. This seems to be mainly because 34% of the Wine-Searcher scores actually come from the Wine Spectator magazine (or 32% if we include all of the database scores), and its contributors produce relatively few scores in the 98-100 range (0.07%). The Wine Enthusiast is the next-biggest contributor (17% of the scores), and even it has only 0.16% of its scores in the 98-100 range.

Finally, it is instructive to look at the four Australian critics plus the lone New Zealander. James Halliday has long been singled out in Australia for handing out a lot of high scores (eg. What's in a number? Part the second), and the data show that he does indeed use 100 points more than do any of his compatriots. However, the 98-100-point data show a very different picture. Jeremy Oliver is the only one with fewer 98-100-point wines than Halliday; and Huon Hooke and Bob Campbell exceed Halliday to the tune of 3.0 and 3.6 times as many wines! Apparently, only Oliver has not yet succumbed to the lure of the high-level scores.