The modern ranking of Bordeaux left-bank wines

A few weeks ago, I looked at what various modern commentators have written about how modern Bordeaux wines compare to the official 1855 Bordeaux Classification of the Médoc châteaux (Making modern sense of the 1855 Bordeaux classification). In that post, I looked only at the five groupings of the châteaux — Premier Crus Classés through to Cinquièmes Crus Classés (First to Fifth Growths),

However, classifications are artificial, because they usually have arbitrary boundaries between the groups. In this case, wine quality varies continuously, and the best wines in one group are therefore not all that different from the worst wines in the next higher group. As I noted in the previous post, no-one wants to be the top château in class 3 instead of the bottom château in class 2.

This issue can easily be circumvented for the Médoc wines, because the original assessment, by the Bordeaux Brokers’ Association, actually ranked the top Haut-Médoc châteaux (as well as putting them into groups). This ranking was intended to be based on quality, as determined by the long-term prices achieved by the various wines.

Obviously, this process can be repeated in the modern world; and so it comes as no surprise that various people have done so. I will be looking at their results here.

Modern ranking schemes

I have compiled the data from a number of modern rankings. These are attempts to rank the châteaux anew, in their modern context, without direct reference to the ranking of 1855. However, it is worth noting that this procedure is somewhat confounded by the fact that the current prices are influenced by the 1855 ranking in the first place.

In all cases, these rankings are derived from a calculated score of some sort.

1. Gary M. Thompson and Stephen A. Mutkoski (2011) Reconsidering the 1855 Bordeaux classification of the Médoc and Graves using wine ratings from 1970-2005. Journal of Wine Economics 6: 15-36. The list was based on the quality scores from three expert wine ratings (Wine Advocate, Wine Spectator, International Wine Cellar). The châteaux ranking was then derived from a regression analysis incorporating a number of relevant factors.
2. Albert Di Vittorio and Victor Ginsburgh (1996) Pricing red wines of Médoc vintages from 1949 to 1989 at Christie's auctions. Journal de la Société Statistique de Paris 137: 19-49. Based on auction sales prices from 1980–1992. The châteaux ranking was derived from a regression analysis incorporating a number of relevant factors.
3. (a) Orley Ashenfelter (1988) A new and objective ranking of the chateaux of Bordeaux (based on auction prices, of course!). Liquid Assets 5: 1–9. (b) Orley Ashenfelter (1997) A new objective ranking of the chateaux of Bordeaux. Liquid Assets 13: 1-6. These two lists were based on average auction sale prices from 1985–1988 and 1994–1996, respectively. I averaged the two listed scores for each château.
4. The Liv-ex Fine Wine 100 Index tracks the trading-exchange prices of 100 top wines (95% of which are from Bordeaux). Rankings of the Bordeaux châteaux, based on average prices over the previous 2-5 years, were produced in 2009, 2011, 2013, 2015, and 2017. I have taken the average of these list prices. Sadly, the 2017 list has only 48 châteaux, not 60, with only five 5th growths listed — the text says “5eme – £250 to £312” but the list has nothing < £274.

This provides us with nine separate rankings of the wines, based on:

1. auction prices 1980–1992 2. auction prices 1985–1988 3. auction prices 1994–1996 4. wine-exchange prices 2003–2007 5. wine-exchange prices 2005–2009

6. wine-exchange prices 2007–2011 7. wine-exchange prices 2009–2013 8. wine-exchange prices 2016–2017 9. wine-quality ratings 1970–2005

Obviously some of these price periods overlap, which will give extra weight to data from the duplicated years. Also, the ninth ranking is based on expertly assessed quality, which is only loosely related to price. Nevertheless, combining these rankings still seems like a worthwhile exercise; and so this is what I will do below.

Another issue is: exactly which châteaux were included in each ranking. For my purposes here, the number varies from 58–68, out of a total list of 70 châteaux. The differences stem from the slightly different purposes of the authors, and what data were available to them. There were 61 châteaux from the original 1855 list (it started at 58, in 1855, but some estates have been sub-divided and others combined), plus nine extra châteaux that make it into the rankings of various of the researchers, based on the modern quality of the wines.

Comparing the rankings

So, when comparing these nine modern rankings, the number of times each château has been ranked varies. Two-thirds of the châteaux (46 / 70) appear in all of the rankings, and 65 / 70 appear in more than half of the rankings, but some château appear only rarely (eg. the small estate Château Desmirail appears only twice, even though it was included in 1855).

In order to address these inequalities, each ranking scheme was converted to a proportion from 0 to 1, before being compared to the other rankings. The ranking schemes can then be compared most easily using the standard Correlation coefficient.

First, the correlations among repeat rankings by the same people are all >80%, and actually go up to 97% (Liv-ex 2013 versus Liv-ex 2015). So, at least we can say that the researchers are consistent through time — once they get a ranking they pretty much stick to it!

Second, in most cases the correlations between people are much lower, in the range 50–70%. This is still quite high, as a half to two-thirds of the variation in rankings is shared between the researchers. The cause of the variation that is not shared is not discernible in any detail, other than to say that the different sources of prices clearly have some effect.

The main exceptions to this point are the correlations between the ranking of Di Vittorio & Ginsburgh and the two rankings by Ashenfelter, which are 91-93%. These are based on auction prices covering the years 1980 to 1996, and so it is perhaps no surprise that they produce very similar rankings of the châteaux. Oddly, though, the correlation between the rankings of Ashenfelter 1988 and 1997 is only 83%, indicating that there are numerous small changes in the rankings over the decade separating the two.

Of these modern rankings, we might expect that the Ashenfelter and Di Vittorio & Ginsburgh rankings would most closely match the procedure allegedly used by the Bordeaux Brokers’ Association to produce the 1855 ranking. However, this does not mean that they closely match that ranking. Indeed, the correlations of the modern rankings with the 1855 ranking are mostly 30–40% (actual range 29–47%). That is, only about one-third of the variation in today's rankings matches the situation in 1855 — there is much more difference than there is similarity.

The modern ranking

So, what might a new ranking look like? Given that there is more similarity than difference among the modern rankings, we could simply combine them, by calculating the average rank for each château across the nine rankings. I have included the final list at the bottom of this post.

We can get a feel for this combined ranking by using a simple stripe plot, as show next. Each vertical stripe represents one of the 70 châteaux, with the average-rank scores increasing from left (best) to right (last).

Based on what we see here, we could, if we wanted, group the châteaux, although it would not be into five classes, as was done in 1855. Three main groups would be practical, although the second and third of these could then be further sub-divided.

There would be 13 châteaux in the top class, with no distinction between First Growth and Second Growth, as there was in 1855 (five First and 14 Second). This seems to accord with modern opinion, that the best of the Seconds are not less than the Firsts.

There are also three châteaux at the bottom of the ranking that probably should not be included in the list, as being consistently inferior these days: Château Cos Labory, Château Belgrave, and Château Lynch-Moussas.

Finally, we can visually compare this modern ranking with that of 1855. The dots at the left represent the châteaux ranks in 1855 (from bottom to top), and these are connected by lines to their position in the 2019 ranking. The gaps in the 2019 ranking represent those châteaux that were not ranked in 1855, while the groups of 2–3 dots in the 1855 ranking represent those properties that have since been sub-divided.

Note that at the top of the list (the bottom of the figure) there have been only local changes in the ordering. However, elsewhere there have been wholesale changes in rank order. As noted above, there is much more difference than there is similarity in the two lists.

Of course, who knows what will happen next, if global climate change has the expected effects on the wine industry. Check with your local wine blogger in 10 years’ time.

---

Ranking of the Médoc châteaux
based on the average ranks from the nine modern rankings

Château Latour
Château Lafite-Rothschild
Château Mouton Rothschild
Château Margaux
Château Haut-Brion
Château La Mission-Haut-Brion
Château Palmer
Château Léoville-Las Cases
Château Cos d'Estournel
Château Ducru-Beaucaillou
Château Pichon Longueville Comtesse de Lalande
Château Montrose
Château Lynch-Bages
—————————————————————
Château Pichon Longueville Baron
Château Pape-Clément
Château Léoville-Barton
Château Beychevelle
Château Léoville-Poyferré
Château Haut-Bailly
Château Calon-Ségur
Château Gruaud-Larose
Château Grand-Puy-Lacoste
Château Pontet-Canet
Château Rauzan-Ségla
Château La Lagune
Domaine de Chevalier
Château Lascombes
Château Branaire-Ducru
Château Smith-Haut-Lafite
Château Duhart-Milon-Rothschild
Château Giscours
Château Saint-Pierre
Château Talbot
Château Malescot St. Exupéry
Château Brane-Cantenac
Château Langoa-Barton
Château Chasse-Spleen [doubtful position due to few data]
Château Clerc-Milon
Château d'Issan
Château Cantenac-Brown
Château d'Armailhac [aka Mouton Baronne Philippe]
—————————————————————
Château Lagrange
Château Cantemerle
Château Haut-Batailley
Château Boyd-Cantenac
Château Prieuré-Lichine
Château Batailley
Château Gloria
Château Kirwan
Château Rauzan-Gassies
Château Haut-Marbuzet
Château Durfort-Vivens
Château Ferrière
Château Malartic-Lagravière
Château Grand-Puy-Ducasse
Château Haut-Bages-Libéral
Château Marquis de Terme
Château Lafon-Rochet
Château Pédesclaux
Château Marquis d'Alesme Becker
Château Pouget
Château du Tertre
Château Croizet Bages
Château La Tour-Carnet
Château Desmirail [doubtful position due to few data]
Château Dauzac
Château de Camensac
—————————————————————
Château Cos Labory
Château Belgrave
Château Lynch-Moussas