Monday, April 13, 2020

The Wine Spectator Top-100 lists from 1988 to 2019

The Wine Spectator produces an annual Top 100 List, which is intended to include what they perceive to be the best 100 wines of those that they have reviewed during the previous 12 months “based on quality, value, availability and excitement”. A few weeks ago I asked: Is there value-for-money in the Wine Spectator Top 100 List? My answer was that there was both good value and poor value among the wines in the list for 2019.

It is now time to look at all 32 of the Lists from 1988 to 2019, inclusive. How consistent has the scoring system been? How has the cost of the wines changed? Has the same value-for money-pattern been maintained through time?


Let's start with the quality scores. The first graph shows the average scores throughout the 32 lists — each point represents the average of the 100 wines (vertically), for each of the 32 years (horizontally).


As you can see, these averages are not exactly random. There seems to be a distinct decrease through the first 9 years, followed by an increase until c. 2004, after which the average has remained steady. That is, with three exceptions (highlighted in pink: 1998, 2000, 2006), the pattern seems to be as shown in the next graph.


We should be wary of when studying time trends (Do winery quality scores improve through time?). We first need to ensure that the data are comparable between the different years, before we start comparing those years. This time pattern may have to do with exactly who provided the quality assessments for each year. The wine scores are a mixture from different tasters within each year, and obviously the group of tasters has also changed through time. [Addendum: Miquel Hudin has a suggestion in the Comments below.]

However, I have noted before that there is a distinct time trend in Wine Spectator quality scores that does not exist for most other wine publications (The Wine Spectator prefers modern wine styles). Indeed, for the Penfolds Grange Bin 95 wine that I analyzed in that previous post there is the same dip in the scores until 2005, followed by a plateau. Is this a coincidence?

Alternatively, it might be that the criteria for inclusion in the annual lists have changed through time. Currently, inclusion of a wine is “based on quality, value, availability and excitement” — the latter, in particular, is a bit nebulous, and may be subject to time trends.

We can now move on to the average prices of the wines, as shown in the third graph, where each point represents the average price of the 100 wines (vertically), for each of the 32 years (horizontally). Here, there is a clear increase through time, as shown by the dashed line on the graph.


The linear increase accounts for 48% of the variation in price, which is the sort of thing that might be expected from inflation. The price variation from year to year (the other 52%) presumably reflects the different inclusion criteria used in different years.

However, the increase in average price is less than the inflation rate during the 32 years. According to the US Inflation Calculator, the average inflation rate from 1988–2019 was 2.5% per year, whereas the wine prices in the graph increase by an average of only 1.5% per year. That is, based on inflation, the wine prices would have more than doubled during the 32 years: eg. average of $35 to $75. Instead, the graph shows a 50% increase (to c. $50).

Does this mean that wine has become better value for money over the past three decades? After all, value for money is probably more important than a wine score or a wine price!

We can look at this question in the same manner as I did in the previous post on the Top 100 Lists, by graphing the prices of all 3,200 wines versus their quality score, as shown in the final graph (one dot per wine). In all cases, I have adjusted the prices for inflation, so that they are all compared at 2019 prices. To keep the graph manageable, I have not shown one of the wines, costing $850 and scoring 100 points.

Value-for-money in the Wine SpectatorTop-100 list 1988-2019

This graph is very similar to the graph for 2019 that I showed in the previous post. Indeed, the simple exponential economic model shown as the solid line has a correlation of 64%, compared to 69% for 2019 alone. This means that roughly the same value-for-money relationship has been maintained across the 32 years, which is quite remarkable.

So, the wines that are far above the model line are poor value for money, while those below the line are much better value. It seems that 98 points has a lot of good-value wines (ie. < $50). There are a couple of very poor-value wines: 92 points for $245, 93 points for $220.

The five outlying points at the top-right, plus the dot not shown, are in a world of their own, pricewise. As I have pointed out before, there are several different categories of wine price (How many wine prices are there?), and therefore trying to combine normal and luxury wines in the same analysis is not likely to work (Luxury wines and the relationship of quality to price).

Finally, there is another way to look at variation in value for money, which is to take into account the wine’s origin. Some time ago, Thomas Girgensohn posted on his blog a discussion of Price vs. quality - an international comparison, using the Wine Spectator summary data for 2018:
At the end of each year Wine Spectator publishes information about their ratings by country and average price. Below, I have graphed this information [for 2018] to show where the bargains are, and where you pay too much.
His results are quite interesting, so check out his graph.

7 comments:

  1. I think it's pretty easy to explain the dip, rise, and plateau. Thomas Matthews came on as Executive Editor in 1999. From this point, there was a marked rise and then consistent-ish scores overall as he has remained in this position.

    You can see this as the voice of the publication being constant, or you can see it as someone being staid in their ways and not really changing opinions even though the wine world has changed massively over the last two decades. I vote more for the latter as it's seen in other publications as well, especially in the US.

    Miquel Hudin
    www.hudin.com

    ReplyDelete
    Replies
    1. Thanks for the information, which makes perfect sense. This would also explain why the Penfolds wine follows the same pattern.
      David

      Delete
    2. Yes, makes perfect sense. In addition, We'd like to see future analyses include the winemaker. As Hudin noted, the wine world is changing greatly of late. We have been tracking the percentage of women winemakers among the Top 100 wines for several years, and there were very few. In the past few years, however, that percentage has somewhat increased. Although wine knows no gender, given the traditional history of winemaking and its being a male-dominated field, the women who become lead winemakers have to be particularly talented to achieve their positions. Illuminating who they are would be of considerable interest.
      Lucia and John Gilbert, Professors Emeriti, Santa Clara University
      https://webpages.scu.edu/womenwinemakers/

      Lucia Albino Gilbert, PhD
      Professor Emeritus, Santa Clara University and The University of Texas at Austin

      Newly released book: Women Winemakers: Personal Odysseys
      Research website: www.womenwinemakers.com

      Delete
    3. Indeed, winemaker would be an interesting piece of information, along with who the actual reviewers are. Gender is, of course, one factor, but there are others as well, including whether a family tradition is being followed.

      Delete
  2. I'd like to see an analysis based on quantity produced. It's commonly noted that their top wine is never a small-production, hard-to-find wine. Always something produced in commercial quantities.

    ReplyDelete
  3. David I have been thinking about pulling this data for a few years now, so thanks for doing this! One interest that I have is regional and producer representation. Do you have any interest in collaborating on a paper on aspects of the top 100 lists?

    ReplyDelete
    Replies
    1. This sounds interesting. You can send me an email direct (winegourd@gmail.com).

      Delete