tag:blogger.com,1999:blog-1392866426745021699.post4515163349080285405..comments2024-03-29T15:15:29.280+01:00Comments on The Wine Gourd: Are wine scores from different reviewers correlated with each other?David Morrisonhttp://www.blogger.com/profile/11578729952036086391noreply@blogger.comBlogger10125tag:blogger.com,1999:blog-1392866426745021699.post-43689276106533911552023-10-07T09:56:48.106+02:002023-10-07T09:56:48.106+02:00Dear Anonymous:
I wrote above: "The next ste...Dear Anonymous:<br /><br />I wrote above: "The next step would be [to] have access to same wine bottle reviewers and see if there is general congruence or incongruence among the reviewers."<br /><br />I invite you to visit these Wine Gourd blog posts:<br /><br />"Laube versus Suckling — their scores differ, but what does that mean for us?"<br /><br />URL: http://winegourd.blogspot.com/2018/03/laube-versus-suckling-their-scores.html<br /><br />"Laube versus Suckling — do their scores relate to wine price?"<br /><br />URL: http://winegourd.blogspot.com/2018/04/laube-versus-suckling-do-their-scores.html<br /><br />Two Wine Spectator reviewers sampled in real time from the same bottles of wine -- and came to diametrically opposite conclusions about relative "quality" (personal preference?) on a number of well-regarded / well-reviewed submissions.<br /><br />With apologies to Winston Churchill, this was not Wine Spectator magazine's "finest hour."<br /><br />It debunked the myth about their 100-point wine scale "methodology" being rigorously adhered to by their reviewers.<br /><br />The rating point difference on some wines was stunningly wide.<br /><br />(A "one-off" circumstance? No. I invite you to visit this Wine Gourd blog post: "How much difference can there be between critics?"<br /><br />URL: http://winegourd.blogspot.com/2019/02/how-much-difference-can-there-be.html)Bob Henryhttps://www.blogger.com/profile/09099196210297757292noreply@blogger.comtag:blogger.com,1999:blog-1392866426745021699.post-87242504385434825062023-10-06T20:10:50.497+02:002023-10-06T20:10:50.497+02:00Yes Bob Henry there is a need for an industry-wide...Yes Bob Henry there is a need for an industry-wide metacritic that normalizes these scores! How great would that be - if they only take reputable critics’ scores, normalize them and then (weighted?) average them. And include it on wine-searcher ⭐️⭐️⭐️⭐️⭐️ (I give that idea 5 stars 😉😉)Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-1392866426745021699.post-83751159770876075242020-02-10T10:59:05.044+01:002020-02-10T10:59:05.044+01:00To the best of my memory, there has never been bee...To the best of my memory, there has never been been awarded a "100-point" score to a dry nonsparkling rosé.<br /><br />"Why"?<br /><br />Well, citing Robert Parker and his 1989 interview with Wine Times magazine (later rebranded Wine Enthusiast), such wines do not improve with age in the bottle.<br /><br />Hence they garner no "bonus" points that place the wine somewhere between 91 points and 100 points.<br /><br />Consequently they bump up against a "glass ceiling" of 90 points.<br /><br />And yet, we have these words decades later from Robert Parker about tasting the "best example" of a particular wine and having an obligation to award it a "perfect score."<br /><br />Is a "perfect score" from Parker 90-points or 100-points?<br /><br />(At the time of his 1989 interview, a "perfect score" for cru Beaujolais was "90 points." And he had never given one . . . until a few years later, when he exceeded his 90-point glass ceiling with reviews on the stunning 2009 vintage cru Beaujolaises in the low to mid-90s.)<br /><br />EXCERPTS FROM THE DRINKS BUSINESS<br />(May 7, 2015):<br /><br />"[ROBERT] PARKER [SAYS]: NOT AWARDING 100 POINTS 'IRRESPONSIBLE';<br />Wine critics who fail to give perfect scores are 'dodging responsibility' according to the world’s most influential wine reviewer, Robert Parker."<br /><br />URL: https://www.thedrinksbusiness.com/2015/05/parker-not-awarding-100-points-irresponsible/<br /><br />By Patrick Schmitt<br /><br />"During an interview with the drinks business earlier this year, Parker -– who developed the 100-point rating system – expressed his urge to award full marks to great wines, and his dismay at those who don’t.<br /><br />"'When, in your mind, the wine is the best example you have ever tasted of this particular wine, you have an obligation to give it a perfect score,' he told db.<br /><br />"On the other hand, he branded those who are incapable of awarding a perfect score 'irresponsible'.<br /><br />"'I think the person who can’t give 100 is really dodging responsibility, because there’s no way they haven’t tasted a wine that is the best example they have tasted from this producer, the best example they could ever think of.”'<br /><br />"He then stated, 'I think it’s irresponsible not to give a perfect score if you think the wine is perfect.'"Bob Henryhttps://www.blogger.com/profile/09099196210297757292noreply@blogger.comtag:blogger.com,1999:blog-1392866426745021699.post-42054254447232891822020-02-10T10:39:18.202+01:002020-02-10T10:39:18.202+01:00Good morning, Michael.
From a statistical perspec...Good morning, Michael.<br /><br />From a statistical perspective, a score of “92 points” is no different from a score of “94 points,” given the ± 2 points to ± 4 points margin of error found by winemaker/scientist/statistician/emeritus professor Robert Hodgson.<br /><br />Chronicled in the pages of The Wall Street Journal by Caltech lecturer Leonard Mlodinow.<br /><br />Excerpts from The Wall Street Journal “Weekend” Section<br />(November 20, 2009, Page W6):<br /><br />“A Hint of Hype, A Taste of Illusion;<br />They pour, sip and, with passion and snobbery, glorify or doom wines.<br />But studies say the wine-rating system is badly flawed.<br />How the experts fare against a coin toss.”<br /><br />URL: http://online.wsj.com/article/SB10001424052748703683804574533840282653628.html<br /><br />Essay by Leonard Mlodinow<br /><br />". . . what if the successive judgments of the same wine, by the same wine expert, vary so widely that the ratings and medals on which wines base their reputations are merely a powerful illusion? That is the conclusion reached in two recent papers in the Journal of Wine Economics [by Robert Hodgson when he analyzed the judging at the California State Fair wine competition, 'North America’s oldest and most prestigious.']<br /><br />. . .<br /><br />"The results astonished Mr. Hodgson. The judges’ wine ratings typically varied by ± 4 points on a standard ratings scale running from 80 to 100. A wine rated 91 on one tasting would often be rated an 87 or 95 on the next. Some of the judges did much worse, and only about one in 10 regularly rated the same wine within a range of ± 2 points."Bob Henryhttps://www.blogger.com/profile/09099196210297757292noreply@blogger.comtag:blogger.com,1999:blog-1392866426745021699.post-13678143245633595552020-02-10T10:04:31.566+01:002020-02-10T10:04:31.566+01:00Differences in the absolute points scale are impor...Differences in the absolute points scale are important, but equally important is potentially non-linear relationships. Maybe "extraordinary" is sometimes equal to "classic" and maybe sometimes it isn't — it may depend on the style of wine, for example. For me, one obvious example is rosé wine, which most reviewers score rather low, in my opinion — I therefore need to adjust my interpretation of the scale accordingly.David Morrisonhttps://www.blogger.com/profile/00276520192744208262noreply@blogger.comtag:blogger.com,1999:blog-1392866426745021699.post-35996575153404492062020-02-10T09:59:15.695+01:002020-02-10T09:59:15.695+01:00Thanks for your comments. Word descriptions of win...Thanks for your comments. Word descriptions of wine have rarely been of help to me, personally, because it is hard to work out what the writer actually means. In that sense, scores are potentially better. The issue, as I see it, is that points are personal, and if I don't know the person then their points don't help, either.<br /><br />One of the interesting thing about today's comparison is that the scores correlate reasonably well but one lot is half a point higher than the other. That helps me interpret the scores, should I wish to buy any of the wines they recommend.<br /><br />One day, I may yet be able to buy wine in Sweden based on the reviewers!David Morrisonhttps://www.blogger.com/profile/00276520192744208262noreply@blogger.comtag:blogger.com,1999:blog-1392866426745021699.post-9263534398133038372020-02-10T09:39:10.267+01:002020-02-10T09:39:10.267+01:00*Historical footnote. The Wine Spectator has revi...*Historical footnote. The Wine Spectator has revised its 100-point scale over time.<br /><br />Citing their June 30, 1994 issue:<br /><br />URL: https://backissues.com/cgi-bin/backissues.cgi?mid/WS19940630.JPG<br /><br />URL: https://backissues.com/issue/Wine-Spectator-June-30-1994<br /><br />95 – 100: Classic<br />90 – 94: Outstanding<br />80 – 89: Good to very good<br />70 – 79: Average<br />60 – 69: Below average<br />50 – 59: Poor, undrinkable, not recommended<br /><br />And now:<br /><br />95-100: Classic<br />90-94: Outstanding<br />85-89: Very good<br />80-84: Good<br />75-79: Mediocre<br />50-74: Not recommendedBob Henryhttps://www.blogger.com/profile/09099196210297757292noreply@blogger.comtag:blogger.com,1999:blog-1392866426745021699.post-91517986904397593142020-02-10T09:38:51.928+01:002020-02-10T09:38:51.928+01:00Is a wine rated “classic” [95-100] by the Wine Spe...Is a wine rated “classic” [95-100] by the Wine Spectator the equivalent of a wine rated “extraordinary” [96-100] by The Wine Advocate? Equivalent of a wine rated “extraordinary” [95-100] by Stephen Tanzer? Equivalent to a wine rated “classic” [98-100] by Wine Enthusiast?<br /><br />Is a wine rated “outstanding” [90-94] by the Wine Spectator the equivalent of a wine rated “outstanding” [90-95] by The Wine Advocate? Equivalent of a wine rated “outstanding” [90-94] by Stephen Tanzer? Equivalent to a wine rated “excellent” [90-93] to even “superb” [94-97] by Wine Enthusiast?<br /><br />Is a wine rated “very good” [85-89] by the Wine Spectator the equivalent of a wine rated (at the upper end of the range) “very good” [80-89] by The Wine Advocate? Equivalent of a wine rated “very good to excellent” [85-89] by Stephen Tanzer? Equivalent to a wine rated “excellent” [87-89] by Wine Enthusiast?<br /><br />If we say “yes,” then let’s take these reviewers literally at their word and match up those words when assessing quality levels across reviewers?<br /><br />The next step would be have access to same wine bottle reviewers and see if there is general congruence or incongruence among the reviewers.<br /><br />[Comment continued below.]Bob Henryhttps://www.blogger.com/profile/09099196210297757292noreply@blogger.comtag:blogger.com,1999:blog-1392866426745021699.post-33190238530380857212020-02-10T09:38:35.770+01:002020-02-10T09:38:35.770+01:00David writes:
“The main source of confusion is th...David writes:<br /><br />“The main source of confusion is that the scores are numbers but they do not have any coherent mathematical properties. This issue does not occur for word evaluations of wine quality, of course.”<br /><br />Can we jettison the numbers and devise a “Rosetta Stone” to compare the relative quality of wines solely connoted by words?<br /><br />Each of these U.S. wine reviewers (individuals or publications) use a scoring scale with six discrete ranges.<br /><br />Citing the Wine Spectator [*] (https://www.winespectator.com/articles/scoring-scale):<br /><br />95-100: Classic<br />90-94: Outstanding<br />85-89: Very good<br />80-84: Good<br />75-79: Mediocre<br />50-74: Not recommended<br /><br />Citing The Wine Advocate (https://www.robertparker.com/ratings):<br /><br />96-100: Extraordinary<br />90 - 95: Outstanding<br />80 - 89: Barely above average to very good wine<br />70 - 79: Average<br />60 - 69: Below average<br />50 - 59: Unacceptable<br /><br />Citing Stephen Tanzer’s International Wine Cellar (https://www.wine-searcher.com/critics-11-stephen+tanzer):<br /><br />95–100: Extraordinary<br />90–94: Outstanding<br />85–89: Very good to excellent<br />80–84: Good<br />75–79: Average<br />70–74: Below average<br /><br />Citing the Wine Enthusiast website (https://www.winemag.com/2010/04/09/you-asked-how-is-a-wines-score-determined/):<br /><br />98-100: Classic.<br />94-97: Superb<br />90-93: Excellent<br />87-89: Very Good<br />83-86: Good<br />80-82: Acceptable<br />Wines receiving a rating below 80 are not reviewed.<br /><br />[Comment continued below.]Bob Henryhttps://www.blogger.com/profile/09099196210297757292noreply@blogger.comtag:blogger.com,1999:blog-1392866426745021699.post-13309887727301205362020-02-10T02:34:48.487+01:002020-02-10T02:34:48.487+01:00"This issue does not occur for word evaluatio..."This issue does not occur for word evaluations of wine quality, of course."<br />Words are a horrible way to express wine quality. Terse descriptions ("great" vs. "wonderful") are much less useful than 92 points vs. 94 points. Long-form descriptive paragraphs are even worse as the same text can describe wildly different quality levels and they're unusably inconvenient for consumers.<br /><br />Numbers, even with imprecision, are a great way to compare two products at a glance. Consumers see points and stars for almost every product category and don't ponder the ontological dilemma of what it means to be a number. 4.3 stars is better than 4.1 stars and is much better than 3.9 stars. 7.2 IMDB score is better than 6.9 - all things being equal, let's watch the 7.2.<br /><br />I just don't see "Confused about difference between 92 and 91 points" showing up much in consumer complaints about wine.<br /><br />However, there must be consumer confusion due to easy/hard scorers. For a project I'm working on, I have about a million professional reviews and have to normalize scores (which I bucket on region/price band - close enough for my needs). Of course a Decanter or Suckling is going to score much higher than a Vinous or Burghound. Indeed, it's an intentional act for newer reviewers who understand they profit from publishing high scores.<br /><br />Is there a need for an industry-wide metacritic that normalizes these scores? Maybe, but I still haven't seen evidence that ratings are a top concern for consumers. <br /><br />PS: Thanks so much for doing this analysis. I love this stuff!Michael Brillnoreply@blogger.com