Quantifying value-for-money wines - part 1

This is the first of a four-part set of blog posts looking at how we might identify value-for-money wines. The topic is not as simple as we might like.

Quantifying value-for-money wines - part 1
   — issues with quantifying value for money
Quantifying value-for-money wines - part 2
   — empirically comparing wines within a specified wine group
Quantifying value-for-money wines - part 3
   — formulae for assessing individual wines against a baseline wine
Quantifying value-for-money wines - part 4
   — empirically comparing wines across wine groups

Issues with quantifying value for money

Good value for money is a rather vague concept, which is presumably why so much has been written about it, often in a rather vague way. So, in order to get started we need to be more specific. Products that are good value for money are often identified using their quality to price ratio (QPR) — high quality for low money = good value. So, for the purposes of these posts I will be talking about QPR.

The principal way of recognizing good QPR is that you expected the product to cost more money than it does, compared to other similar products. QPR is often treated as being a subjective and relative concept. That is, you first have to like the product and, second, everyone’s idea of “value” is likely to be somewhat different. Almost every wine forum has a discussion of the topic (eg. see QPR: fact or fiction?), with almost as many opinions as there are commentators.

This attitude cuts no ice in the world of economics, which is the world inhabited by those people looking for bargains. The basic issue is that subjective assessments of QPR tend to focus on the money not the value. That is, the product must be cheap in order to have good QPR. However, quality costs money, and therefore must be paid for. So, good QPR can involve a lot of money when the quality is high enough. Furthermore, many people think that you have to actually like the product, or it isn't good value. However, good value exists even if it is not valuable to you personally.

The quality : price ratio

Therefore, it should be possible to quantify QPR, at least approximately, so that rational purchasing decisions can be made. QPR can never be exact, of course, because there can be no precise measure of quality, nor will there be a precise price to be paid (price varies through time and space). However, lack of exactness should not stop us from producing a suitable approximation, from which we can make decisions in practice. In the wine world, this means choosing a wine for reasons other than that you simply like the label!

So, QPR needs to be an actual number. In any normal business it should be possible to get that number by dividing some measure of quality by some measure of the price. That way, large QPR values would represent good value-for-money and small values would not. Unfortunately, the wine business is not normal, by any means, and thus measures of quality and price can be rather tricky.

We can leave aside price for the moment. Price obviously varies from country to country, and from state to state within any one country. Price also varies with currency fluctuations, since an awful lot of wine in any given country is imported (see the post on Global wine imports). There is little we can do about price variation.

Measuring quality

Quality, on the other hand, is a very tricky thing for wine. Wine is an "experience good", and therefore perceived quality is tied up with personal experience. Nevertheless, there are plenty of ways to measure wine quality, including scales using three glasses, five stars, and points with a maximum of 10, 20 or 100. Sadly, the only thing these have in common is that more glasses / stars / points is intended to mean more quality.

I won't go into detail here, but these scoring schemes can show a lot of variation, resulting from the fact that they are all based on personal assessment when tasting the wine. These assessments can differ due to: varying preference or experience of the assessors; different expectations among the assessors; varying sensory perception of any one assessor at different times; different environmental circumstances of the tastings; variation among bottles of the same wine; different interpretations of how to apply the scoring system being used; and so on. So, any given quality score is potentially not directly comparable to any other score.

One possible way to deal with this issue is by combining scores from as many assessors as possible. The main trick when doing this is to know how the different assessors have used their scoring schemes — we can't just average the raw scores, because 16/20 for one person is not necessarily the same as 16/20 for the next person. The scores thus need to be standardized before averaging — I have briefly discussed this topic for Bordeaux wines (see the post on Luxury wines and the relationship of quality to price). A subsidiary issue is how many scores must be combined to get a meaningful average score. Two is the minimum, but more is obviously better.

Clearly, the 3-glasses and 5-stars scoring schemes are very coarse (there are not many levels), and they are not likely to give us much variation in QPR (most wines tend to be assigned the middle level). Nevertheless, in these cases the QPR is straightforward to calculate, and it can therefore provide some guidance for purchasing.

Unfortunately, this is not so for the point-scoring schemes. This is because there are minimum values below which scores do not occur. For example, the 100-point scheme has a minimum possible score of 50 (not 0); and below a score of 70 the fluid is not drinkable as wine (although it may be very good wine vinegar). Indeed, you will rarely encounter a wine score below 80 — even the cheapest wine widely available in the USA (the infamous Two Buck Chuck, produced by Charles Shaw winery) scores an average of 80 points on the CellarTracker site. So, the 100-point scale is effectively a 20-point or 30-point scale, in practice.

In the same vein, there is also a minimum price below which wines do not normally occur in practice. For example, in the USA the Two Buck Chuck wines cost US$ 2–3, depending on the state.

So, these non-zero minimums for quality and price mean that simply dividing the quality score by the price will produce a meaningless QPR score for wines. The QPR would look like it was measured in points per dollar (euro, pound ...), but that would not be how it actually is.

The POP score

This issue has been partly addressed by the Liv-ex wine marketplace, which monitors the global auction market. They have devised their POP score, which they describe as follows:

A wine’s POP score is its price-over-points ratio, our loose measure of value. It is calculated by dividing the price of a dozen case of wine by a shortened 20-point score. We have calculated this 20-point score by simply subtracting 80 from the rating, on the basis that any wine under 80 points is unlikely to appear on the auction market.

That is, they subtract a minimum value from the quality score of each wine, before doing the QPR calculation. This addresses the issue of a non-zero minimum score, but not the issue of a non-zero minimum price.

The TBC index

Mike Veseth, at the Wine Economist blog, has proposed dealing with this issue by having a baseline for both quality and price. That is, we subtract a minimum value from both the quality score and the price of each wine, before we do the QPR calculation. He calls the resulting QPR score the Two Buck Chuck (TBC) index. This approach means that any wine with a score of TBC=1 has the same QPR as the baseline wine chosen (ie. the wine with the minimum score and price).

He proposed using 70 as the minimum score when measuring quality on the 100-point scale, and US$2 as the minimum price. So, TBC = (Quality - 70) / (Price - 2). [TBC is thus a bit of a misnomer, since the Charles Shaw wines actually cost US$3 in most places and score c. 80 points.] However, any wine can be chosen as the baseline.

The first graph shows the prices for wines with TBC=1 given the quality=70 and price=2 baseline wine. Any wine whose price and quality places them below the line would be considered good value for money, and anything above the line would be poor value.

Also shown are red dots representing the two wines that Veseth uses as his examples. (1) An 88 point wine for $20 would have a TBC score of (88-70 points)/($20-$2) = 18 points / $18 or a dollar a point — this wine thus lies on the line. (2) An 86 point wine for $10 would have TBC = (86-70)/(10-2) = 16 points / $8 = two points per dollar — this wine lies below the line, and is thus better value for money.

It is important to remember that TBC values depend on the baseline wine, and cannot be compared between different baselines. So, a suitable baseline needs to be chosen for different types of wine, and value-for-money comparisons can only be made within that wine type. The line shown on the graph will move up if a higher price is used for the baseline, and will move to the right if a higher baseline score is used.

The obvious limitation of this simple approach to quantifying QPR is that the relationship between price and quality is assumed to be linear (ie. the graph shows a straight line). However, I have shown in previous blog posts that this relationship is actually exponential (log-linear) — see The relationship of wine quality to price and Luxury wines and the relationship of quality to price. So, the TBC approach is not realistic, because the line in the graph should be a curve.

The Reverse Wine Snob rating system

A similar limitation applies to the QPR rating system used by the Reverse Wine Snob web site. This approach uses a scale of 1-10 for quality (Taste) and 0-10 for price (Cost; a high Cost score represents a low price). The QPR rating is then a combination of these two scores. This approach thus neatly side-steps the non-zero issues raised above, because there is a pre-defined baseline wine.

The RWS-QPR score for any given wine is calculated by starting with a perfect score of 10, for a wine with Taste=10 and Cost=10, and then subtracting 0.25 for each decrease in Cost score and subtracting 0.75 for each decrease in Taste score. A final QPR score > 7 indicates a wine with good quality : price ratio.

This system is illustrated in the final graph. Note that the calculations form a series of straight lines, one for each Cost-score level. This makes clear the linear nature of the RWS-QPR. This may be quite reasonable in this case, because the Reverse Wine Snob site is all about wines with a price less than US$ 20, and within this narrow range the relationship between quality and price may well be approximately linear.

Where to from here?

In the next two posts I will discuss some QPR approaches that address the non-linearity of the quality-price relationship of wines, and thus produce more realistic measures of the quality : price ratio.