Do online wine ratings and searches actually mean anything?

Social media sites like Vivino, Delectable and Cellar-Tracker collate wine ratings from their users, and Wine-Searcher does the same thing for critics, as well looking at the search popularity of wines. These sites sometimes present a compilation of their data as representing things like "the best wine in the world" (eg. Is this the best wine in the world? An app with 35 million subscribers says so) or "the world's most desired wines" (eg. The world's most wanted wines).

This seems like quite a radical conceptual leap, to me. It is one thing to note what wine is, in some sense, the most popular wine, on average, for the restricted set of users represented on any given online site. It is another thing altogether to present this as "the best" in any broader sense. To leap from a restricted user base to the entire world is a form of arrogance, at best, and complete and utter foolishness at worst.

After all, a highly rated wine (in the most general sense) may have little to do with "the best" for anyone other than the people doing the rating. First of all, ratings may have nothing to do with quality, but only with a desire to rate, based on any criteria you wish to name (eg. Should critics rate wine based on environmental impact?) — a rating is more like a popularity contest, rather than a quality evaluation. Second, the raters themselves are rarely representative of any group other than people who wish to provide ratings — they may have any motive at all for doing so. Third, the products being rated are rarely representative of the range of products available — they tend, for instance, to be associated with their snob value rather than their value for money.

So, does the highest average wine-rating on a site like Vivino represent the quality of the wines being rated, or does it represent their snob value? Vivino regularly tells us it is the former, while I suspect it is more likely to be the latter. Does the number of wine-label searches carried out on a site like Wine-Searcher represent the desirability of the wines being searched for, or does it represent their curiosity value? Once again, Wine-Searcher regularly tells us it is the former, while I suspect it is more likely to be the latter.

What use, then, is the information provided by these types of sites? Sure, they are a valuable outlet for the modern penchant for social-media opinions, just like Facebook, Twitter, and YouTube in their respective domains. Given their extensive usage, I presume that these sorts of sites are providing a useful service for Millennials and Generation-Xers — my parents' generation nattered over the back fence or down at the pub, by my children's generation natters online. So, I doubt that we can take these services as anything other than opinions; and certainly not as a source of quantitative information about the actual goods and services, which are hiding firmly in the background. A Twitter storm, for example, has little to do with the topic at hand, but much more to do with human behavior when acting in groups — we learn much more about the people than we do about the topic.

These kinds of observations are a basic tenet of the social sciences. Getting reliable quantitative information out of human beings takes a lot of effort; and it is a topic that has been actively studied for more than a century, without any explicit resolution (see Best practices for survey research). Formally conducted surveys can be useful, although they have their own set of limitations, notably to do with what is known as sampling bias. Social media might seem like a quick and easy way to get at the same type of information; but I doubt that this works. It is more likely "quick and dirty", with the emphasis on the latter; because the "survey" respondents are self-selected (ie. they choose to use the site, and they choose to provide a rating) — this is known to be the worst form of sampling bias.

So, do not allow yourselves to be gulled by pronouncements from online social media sites. They have no more real information about the world than you do — they know only about the opinions of their own users, whoever they may be, and whatever motives they might have.

On a personal note, rarely are the wines commonly mentioned by most of the social-media sites of any relevance to me, in practice. Wine is not for bragging about, but for consuming with dinner (the benefits of which are explained here). As such, value for money is the main information of interest to me prior to a purchase. I will try any wine from anywhere, if there is evidence of it being good value for the money being asked. Most of the wines being bragged about and searched for online are therefore of no practical interest at all. It seems to be a pity that they get most of the attention, while "my type of wine" requires more effort to locate (see Calculating value for money wines). In this sense, the so-called "social media" is often anything but social (and, yes, I am quite well aware that a blog is a form of social media!).