Do different wine critics agree on value for money?

There are only two things I really need to know about a bottle of wine that I am thinking of buying: 1. Is it good value for money? 2. When should I drink it? Let's look at the first question in this post. (Other features, such as possible food pairings, I can work out later.)

I have written several times about evaluating value-for-money, when searching for wines (see "Value for money" under the "Labels For Posts" list at the right of this page. One simple idea is to compare the current bottle price with a quality score from one or more wine commentators.

We all know that different wine tasters can come up with very different opinions (and scores).* However, a somewhat different question is whether they agree on which wines are value for money, irrespective of the actual score they give. This involves the relationship between their points and the cost of the wine — high points for low cost = good value for money.

It therefore becomes relevant to ask whether we would get the same outcome depending on which commentator we choose. The degree of between-commentator variation is another topic that I have discussed before (eg. Are wine scores from different reviewers correlated with each other?  Does the relationship between critics' scores differ by wine type?).

So, it seems reasonable to wonder whether the assessment of value-for-money also differs between commentators. Note that value-for-money is not directly related to price, as expensive wines often get high scores from both professionals and amateurs — these wines may or may not be worth that expense.

To answer this question, I will be comparing two sources of wine scores available here in Sweden, which I have used in posts before: Jack Jakobsson at BKWine Magazine, and Johan Edström at Vinbanken. These commentators are discussed in the post on: Are wine scores from different reviewers correlated with each other?

Here, I will be using the procedure that I outlined in the post on: Calculating value for money wines. For new wines, both of our commentators produce scores out of 20, representing increasing quality, and also report the (standard) bottle prices (in Swedish kronor, SEK). We can simply plot price versus score; and we can calculate the financial cost of each quality point (ie. divide kronor by points, to get a kronor-per-point value).

So, I have gathered the data for last year (2020) from both sources. That particular year was a bit disturbed, of course, resulting in a few data anomalies (see my post How close are repeated wine-quality scores?). However, this should not affect my analysis (when different scores appeared, I simply averaged them).

This process yielded data for 589 wines from Edström and 512 wines from Jakobsson, with 487 of the wines being in common (the commentators could not get to taste all of the released wines, due to Covid-19 restrictions). We could start our analysis by simply looking at the relationship between the scores, for the shared wines. This is shown in the first graph, with each point representing one wine. The pink line would represent equal scores for each wine, while the black line shows the actual relationship.

This shows that the Edström scores are generally higher than the Jakobsson scores, for the same wines. Furthermore, only two-fifths (39%) of the scores are related to each other. This emphasizes that the quality scores themselves do differ between the two critics.

However, our question is about value for money, not the scores themselves. This is plotted in the next graph, encompassing all of the wines. Each point still represents one wine, showing its price (vertically, logarithm scale) and its score (horizontally). The Jakobsson scores are in blue and the Edström scores are in pink. The lines represent the two exponential relationships between price and score. Last year, $US 1 ≈ 9 SEK.

As expected, the Edström points lie to the right of the Jakobsson ones, due to the higher scores. However, the relationship between price and score is not that different between the two critics — about half of the variation in price is related to the score (higher scores generate higher prices). So, it seems that the two critics do have roughly the same opinions in terms of the wines' value for money.

We can look at this directly by calculating the number of kronor that each wine cost per quality point assigned. This is plotted in the next graph, for the shared wines. As usual, each point represents one wine, with the pink line representing equal assessment of value for money by the two critics, and the black line showing the actual relationship.

As you can see, the two critics do, indeed, have almost equal assessments. That is, in spite of their differences in scoring, their assessment of value for money is very similar.

This is very good for me, of course, as a consumer. I can use either critic to identify wines for purchase, if I am interested in trying new wines that are identified as being of particularly good value. This is, of course what I do do — I buy wines that I know and like, and I also buy new wines to try. As noted in the previous post, my preferred "wine recommendation system" is actually people, not a computer (Wine recommenders, and the tyranny of choice). In this case, it will make little difference which person I choose (I just need to allow for the difference in scoring). The best-value wines are simply those in the bottom-left corner of the final graph.

---

* Consider these two reviews, for the Château la Brande Tradition Cabernet Franc 2018 (Fronsac):

Christian Stojkovic:
Full-bodied, great aroma with ripe fruit of cherries, black currants, plums, herbs, oak and coffee, and a good, long aftertaste. In a really good vintage like 2018, Bordeaux can deliver such a good wine for relatively little money. 17 points.

Jack Jakobsson:
Slightly wet dog, young, floral with a little hyacinth, acidic. 14.5 points.