Can we find out whether we really can tell red wine from white?

In an article in the Los Angeles Times way back in 1987, David Shaw noted (He sips and spits — and the world listens) this about Robert Parker:

More than once he [has been] asked if he’d be willing to demonstrate his consistency. Would he taste and score five or six wines “blind” — without knowing what they are — and then taste and score them again a day or two later? “No,” he says. “I'm not doing trained dog tricks. I’ve got everything to lose and nothing to gain.”

It seems that Mr Parker neither respects scientific experiments nor understands their use — experiments are not dog tricks. Sadly, though, this is often the response of experts, who apparently see their authority as being challenged by any objective assessment of their (alleged) expertise.

A more general case of this in the wine industry is the ability to recognize whether a wine tasted blind is red or white. It has sometimes been claimed that most people cannot tell the difference, when the wines are tasted under identical conditions.

This topic is worthy of some discussion, because it seems to me (as a professional scientist) that: (i) it has never actually been shown that people can not tell; and (ii) testing this claim scientifically would actually be rather hard.

Potential problems

Most of the so-called experiments reported in the general media strike me as being rather inadequate. The main issue is comparability. This is a basic requirement of experiments, that the things being compared are directly comparable. If the things being compared differ in several ways, then how can you know which one is causing the main effect?

In order to see the issue at hand, consider this point:

White wines are usually consumed at a lower temperature than red wines. If you compare a red and a white at the same temperature, then one or the other wine is being tasted sub-optimally. Your ability to correctly identify a refreshing white when chilled does not help you recognize that same wine at room temperature, where it may be rather bland.¹ Similarly, that red wine that is so redolent of forest fruits at room temperature may be completely closed if you chill it. How could we make the comparison of reds and whites directly comparable?

Similarly, experience plays a great role in being able to make consistent comparisons. If you have no experience of the comparison, then you may not know what difference you are being asked to detect. For example:

If you are not used to drinking expensive wines (or to having really cheap ones!), how do you recognize their differences? If you have experienced a wide range of wine prices, then you may have a chance, but if your wines usually come out of a cardboard box, then that $100 bottle you were given as a present may seem “different”, but you might not necessarily say that it is “obviously expensive”.²

If you are not familiar with certain wine styles, then how do you recognize their different characteristics? A Barossa Shiraz (from South Australia) is quite different from an Etna Rosso (from Sicily), but if you have never had one or the other, then how can you correctly tell which is which?

This should make it clear that experimental comparisons are tricky things. Most so-called experiments conducted by non-experts therefore usually have holes in them that you could drive a car through. This comes quite simply from not trying to make the comparisons directly comparable, in a way that would answer the experimental question.

Previous reports about wine

Having set the scene, we can look at the media reports.

Far and away the most popularly mentioned work allegedly relating to red wine versus white is reported endlessly by web sites wanting to debunk wine tasting, but also by some of the reputedly more responsible press (eg. You are not so smart: why we can't tell good wine from bad ; Does all wine taste the same?). The work was done by Frédéric Brochet in 2001. He is a French cognitive psychologist studying what he calls “perceptive expectation”. Brochet showed that many people given a white wine that has been dyed red will describe it as they would a red wine; and that the same wine served in two different bottles of supposedly different quality wines will also be assessed differently.

However, none of this has anything to do with telling red wine from white, as explained clearly at the Urban Legends of Science blog (About that wine experiment) — all it does is show that what we see is important for our brain. The visual clue is, indeed, important for wine, which we already know — from professional wine descriptions, if nothing else, but it is also the basis of having black (opaque) tasting glasses. But what has that got to do with telling red from white when are trying to do so? Moreover, there is not enough information in the reports to tell how comparable the comparisons were.

You can read more about the relationship of taste, smell and sight at Slate — Do you taste what I taste? (You might find interesting the discussion of butyric acid, which is in perspiration, vomit and parmesan cheese.³) See also: The taste of wine isn't all in your head, but your brain sure helps and The red and the white — taste is partly expectation. Scientists are, of course, working on this — see: How expectation influences perception.

Moving on to actual blind-tasting comparisons of red wines and white wines, Thomas Matthews at the Wine Spectator reported in 2002 on his own experiment (Can you tell red from white?). There were 6 wines, either red or white, and 7 people tasting, so that there were 42 attempts to “guess right”. He reports 40 correct results (95%). This hardly debunks the idea of telling red from whites!

There is one other report that seems worthwhile, from Erik Rasmussen at his American in Spain blog (Blindfolded wine taste test: can you distinguish white from red? Cheap from expensive?). He chose four wines of different types, as listed in the table below, and four people tasted each wine at room temperature, while blindfolded. Here, the comparison has been taken seriously, because the wines all came from the same region (Rioja), the tasters had reasonable expertise, there were equal sexes, etc.

The table shows the results for the four wines (the rows), the four choices (the columns), and the four tasters (the cell counts). The boxes show the correct guesses (7 / 16 = 44% correct) — because each taster had four choices, there is only a 25% probability of getting a wine right by random chance, and 4% for getting all four right.

In spite of the small experimental sample, the results here are statistically significant (using the Fisher Exact Test). For example, no-one identified the white as a red, and the expensive red was always identified as a red. So, these people, at least, did better than guessing, with three people each getting two wines right. Most interestingly, the ones they got right were generally related to their expertise with the wine styles concerned.

Finally, we could also consider the case of blind winemakers. One such person is David Hunt, from Hunt Cellars (see A man of taste: keen sensory perception and meticulous blending help David Hunt achieve his winemaking vision). The difficulty for our purposes here is that he does know what he is tasting before he tastes it — he just can't use his eyes to see it.

Conclusion

So, we may never know whether we can tell reds from whites, although the work done so far suggests that we can do much better than a random guess. The next time you read a claim that we cannot tell, then you will know that the person is bluffing. Besides, it is quite certain that “tasting” involves our mouth, our nose and our eyes — the brain needs all three to decide what it is drinking, and whether it likes it.

Bonus notes: experimental requirements

Experiments are all about getting the comparisons right. So, what do we need to do?

There are actually three ways that you can try to make your experimental “treatments” comparable:

• Incorporate all of the variation into your comparisons (eg. choose several types of wines, from several regions, and taste them at several temperatures, etc) — this leads to a very large and complicated experiment, but it will be very informative because you have studied everything
• Try to control the conditions to minimize variation (eg. similar wine-making styles, and grapes, similar tasting expertise, all done at the same time) — this usually leads to a small experiment, and the results cannot be generalized beyond the specific conditions that you studied
• Randomize the variation (eg. choose lots of arbitrary wines and temperatures) — you end up with lots of variation in the results, and you thus need a very large experiment.

These approaches are not mutually exclusive, and they may all appear in the same experiment. For example, you might incorporate a few types of variation (eg. temperature and wine style), while you control others (eg. all tasters are experts).

What sorts of variation might we be concerned about when comparing white and red wines? Here are some possibilities:

• grape type — green skins all the way through to dark red skins
• skin contact — for most grapes the color is in the skin, so maceration time is important
• tannin level — some wines are naturally full of tannins, while others receive oak treatment
• fruitiness — some wines are prized for their fruitiness, others for their savoriness
• sweetness — residual sweetness is often preferred by consumers
• level of maturity — young wines are different from mature wines
• quality — quality is usually related to grape-growing and wine-making effort
• temperature — the temperature for optimal sense detection when tasting
• the way we sample the wine — sniff; swirl and sniff; cover, swirl then sniff; sip; swallow.

I am sure that you can all think of more possibilities. The point, though, is how do we deal with these in an experiment, using one or more of the above approaches? You can see that it will not be easy. Indeed, you might consider it to actually be impractical. That is why I have never tried it, for this case.

---

¹ I clearly recall several times drinking white wines in Sicily that became more and more uninteresting as they warmed up on the restaurant table. Fiano wines are very good when kept cool!

² In 2008, the American Association of Wine Economists Working Paper 16 reported:

In a sample of more than 6,000 blind tastings, we find that the correlation between price and overall rating is small and negative, suggesting that individuals on average enjoy more expensive wines slightly less. For individuals with wine training, however, we find indications of a positive relationship between price and enjoyment.

³ I still have a clear memory of a traumatic experience as a child, in which the restaurant waiter put parmesan on my spaghetti, and I couldn't eat it because to me it smelled like vomit.