Precision and accuracy of numbers — getting it right

Every day we are bombarded with numbers, usually from the media. Unfortunately, the people presenting these numbers often do not understand the relationship between the precision and the accuracy of their numbers, and so they are prone to mislead both themselves and their readers.

In my experience as a scientist, this is true even in the professional literature; and my recent experience in the professional literature of the wine world suggests that it is no different there, either. So, it is worth explaining this situation, to see if I can't encourage people to get it right.

The accuracy of a number refers to how close it is to the truth. If I claim that something costs $10 when it actually costs $15 dollars, then I am not being very accurate.

The precision of a number refers to how many digits I am using, or how many decimal places I present. If I claim that something costs $10.11 rather than $10, then I am being more precise (I've used four digits rather than two).

This distinction is often illustrated using the idea of shooting at a target, as shown above. Precision refers to how close together are repeated shots, while accuracy refers to how close the shots are to the center.

A problem occurs when the precision of any number is greater than its accuracy, because that will be misleading. For example, if I claim that something costs $10.11 when it actually costs $15, then the precision of my number (to the nearest cent) gives a spurious sense of accuracy (I am not even accurate to the dearest dollar). This is bad; and it can be easily avoided.

I can illustrate this using the following example from the recent wine literature. In this case, the data summarize some of the characteristics of 48 people who were sampled. When presented as percentages, the numbers cannot be more accurate than to the nearest 2% — after all, the only numbers possible are 0 people out of 48 = 0%, 1 / 48 = 2%, 2 / 48 = 4% .... 47 / 48 = 98%, 48 / 48 = 100%.

However, the numbers as presented in the paper were to the nearest 0.1%, which is 1 out of 1000 not 1 out of 48, as shown in the first table. The 60.4% actually refers to 29 out of 48 people, not to 604 out of 1000. This is misleading.

In this case, presenting the numbers to the nearest 1% (ie. dropping the decimal places) would be better, because the precision would more nearly represent the accuracy.

As an alternative example, the next table shows two different sample sizes, 136 and 50. A sample size of 136 may well justify an accuracy of one decimal place but not 2 such places; and a sample size of 50 probably does not justify even one decimal place. Just because we can calculate a number to many decimal places (lots of precision) does not mean that the accuracy justifies this.

These situations are easy to avoid — precision should simply never exceed the accuracy.

Note: I have not identified the authors of either of the examples illustrated here. I agree with Bjørn Andersen (in his book Methodological Errors in Medical Research) that we should not "pillory a few for errors which many commit with impunity".

Wine writing, and wine books

This blog usually deals with wine industry data in the form of numbers, but there are other forms of data that could be examined instead. One of these is wine tasting notes, which are supposed to provide us with important information about wines.

These wine notes, from critics and commentators, are odd things. They vary from boringly monotonous to verbally flowery, but I am not convinced that they give us much useful information, very often.

Part of the problem seems to be that half of the critics want to be writers, not commentators, and the other half want to be critics but can't write very well. The readers are then caught between the devil and the deep blue sea — they want some useful information, but all they get are words.

As an example of what I mean, take this recent wine tasting note from an Australian wine merchant's web site:

Brilliant very pale straw colour with a greenish tinge around the outskirts and a watery hue. Intense lime, green apple and citrus aromas leap from the glass followed by some distinctive talc notes and orange rind. Racy with great fruit purity, the palate features concentrated flavours of lime and citrus over talc with a steely mineral infused finish. Fresh crunchy acidity with a long aftertaste of lime, citrus, talc and steely mineral. A first class rendition of Polish Hill Riesling.

At the other extreme, here is Philip White, the poet laureate of wine writers, on exactly the same wine:

Pacing from the kitchen to my desk, I snapped the lid of this, and hardly was the cap away and a heady wherrul of the lychees and limes of the Polish Hill River stopped me in the doorway. It overwhelmed the smell of sheep and wet pasture blowing through my windows; cast out the smell of winter. It was as if it had a pump in it, or some magical compression was releasing a heady fresh essence of these fruits. Like, within a metre of pacing air it actually got right up my nose before the lid was properly off. It brought me to a halt. Pour it, and the dusty vintage sky of the old slopes east of Clare; their stubble and stone seem to cover those fruits in a grainy armour. It is a lovely summer smell in all this wintry damp; the paperflowers in warm stoneware. Drink it, and all that simply invades you. It makes me realise why Riesling scares some people. Wine like this is very authoritative. Rare examples like this can be surly organoleptic bullies unless you can handle it right out here on the front.

This is impressive stuff, to be sure, but I feel like I have drifted into a DBC Pierre novel, not a wine column. (Peter Finlay is a friend of White's, which is why I chose this example.)

When I read a wine note I wonder to myself three things:

1. Will I like this wine enough to buy a bottle?
2. When should I drink that bottle?
3. What food should I eat with it?

Most wine notes do not answer even one of these questions, let alone all three of them together, in spite of the plethora of words.

The answer to the first question should be contained in the tasting comments which, in that sense, matter more than any score that might be given. Unfortunately, the answer is rarely contained in the words of the writer. Instead, it is contained in finding a writer who has the same tastes as one's self. This takes a bit of trial and error, but it needs to be done.

Back in the 1980s, in Australia, I learned that suggestions from Alan Young (author of Australian Wines and Wineries) were always worth checking out. James Halliday, on the other hand, gave every Australian wine 90 or 95 points, and so was of little use. Moreover, Jeremy Oliver, seems to have a palate similar to my own, and so his recommendations are helpful to me, as were those of Robin Bradley (compiler of many editions of Australian Wine Vintages) in his day. The Wine Front web site can also be good, but the tasting notes come from any one of three people (Mike Bennie, Campbell Mattinson, Gary Walsh), and their tastes in wine clearly vary.

It can also be helpful to consult the community sites, such as CellarTracker and Vivino, since the comments there come from a range of commentators, rather than a single person who may or may not match my own tastes.

The second question (when to drink it) is important because I do not often buy more than one bottle of any particular wine. I figure that I should try as many wines as I can during my life; and given that there are 120,000 different ones produced every vintage, I am not going to spend much money buying multiple bottles. So, most wine/vintage combinations get one chance and once chance only in my cellar. I therefore need to know about the "cellaring potential" of my wines.

This question, however, is a tricky one, because no-one knows before-hand when a wine will be at its best, or how long it will remain there. But people with more experience than myself might have a better idea than I do. At best, their suggestions can be very helpful, and at worst they usually won't do much harm. I have drunk quite a few of my bottles too soon and some of them a bit late, but I have also drunk innumerable of them when they seemed very good to me. I am therefore very grateful for any advice I can get in this matter. [The first Polish Hill Riesling tasting note quoted above helpfully provided this information: "Drink now or cellar 6-8 years."]

The third question (what to eat with it) is important because I almost always drink wine with food, rather than on its own. This is not necessarily true of most most wine writers of my experience, who routinely attend tastings, or organize their own, where a large number of wines are tasted sequentially. Such wine tastings help you learn a lot about wine, but they teach you nothing about drinking it, because wine is best drunk with a meal.

Such an attitude does not sit well with the likes of Robert Parker, of course, whose taste in wine clearly does not include many wines that go well with food. I want to enjoy my wine at dinner, not arm wrestle with it beforehand, as he does.

So, I choose my meal first, and then I choose a wine to complement it, or enhance it; and thus an indication of which meal would complement which of my wines is a valuable suggestion. For example, in the graphic above concerning the M de Minuty wine, the four symbols at the bottom right provide some easily interpreted suggestions.

Why, then, do most wine writers restrict themselves to giving me little more than a flowery description of the smell and taste of the wine? Surely I can work this out for myself when I drink it?! No amount of adjectival wordplay is going to make the wine taste any different to me, because I have taste-buds of my own. I need expertise, not repartee.

In a similar vein, many wine books often suffer a related problem, but in reverse. Instead of giving me words they give me numbers. But are these numbers any more useful? The numbers provide me with ratings for each winery, ratings for each of their wines, and ratings for each of the vintages of those wines. However, can I answer any of my three questions from these numbers? Not often.

Even when there are words in addition to the numbers, the words usually tell me nothing more than I have indicated above. Mostly, I cannot even tell what style of wine the numbers refer to.

There are exceptions, however. The picture immediately above is from the second edition of The Australian Wine Handbook (1994), by Jeremy Oliver (mentioned above). It presents the key to the numbers contained in the book, and how to interpret them. Along with the Vintage Quality ratings (on a 10-point scale), there is a clear indication of the wine style (Style Code: DR1-7 for seven types of dry reds, DW1-7 for seven types of dry whites, SP1-4 for sparkling wines, SW1-4 for sweet wines, etc), an indication of the anticipated cellaring potential (Maturity Code), and a clear indication of when to drink each vintage. The only thing missing is the food; but I can probably work that out from the wine style.

So, a typical winery entry in that venerable book looks like this.

In this case, the DR3 refers to: "Medium-weight wines with ripe fruit, soft palate and integrated medium tannic finish. The modern early-drinking cabernet-merlot blends, lighter shirazes." The DW2 refers to: "Floral, fragrant, fruit-driven and dry, able to develop great complexity with time. More of a 'riesling' style."

Sadly, this book ceased publication with the third edition (1997). With the subsequent publication of The OnWine Australian Wine Annual (2001) the Style Codes and Maturity Codes were dropped, and the Vintage Quality migrated to a 20-point scale. Subsequent editions of The Australian Wine Annual were similar, but used a 100-point scale, instead. I still miss the lost information.

Some alternative opinions about wine tasting notes

David Farmer

Do tasting notes have any value?
Writing tasting notes about great wine

Kim Brebach

Wine poetry, aroma wheels & perfect scores
Wine writers, tortured prose and hanging offences

Bizarre wine data

While I was looking for some data about wine imports/exports for Australia, I came across a report from the Global Agricultural Information Network, which is part of the Foreign Agricultural Service, of the United States Department of Agriculture.

Among much other data, this report does contain import / export data. Chart 5 claims to show data over four years for the major countries from which Australia imports wine, in units of Millions of Liters. Cross-checking with other sources, the data for New Zealand, France, Italy, Spain, Chile and the United States seem to be correct.

However, the presence of Sweden, Ireland and the Netherlands in the graph seems to be quite bizarre. Indeed, none of these countries even produces the amount of wine shown for each year, let alone exports it to Australia. After all, 10 million litres is a lot of wine for somewhere like Sweden to be exporting to anywhere!

To many people's surprise, Sweden does actually have an active wine industry (see Wikipedia), as also do both Ireland and the Netherlands. There are c. 15 commercial wine producers in Sweden, out of an estimated 250 vineyards. More than 40 vineyards are listed at the growers' association, Föreningen Svenskt Vin; and there is even a book about some of them: Tjugo Skånska Vingårdar (see The vineyards of Sweden). The latest official figures I can find (from the agriculture ministry, Jordbruksverket) are for 2009, when Sweden apparently produced 17,859 litres of wine for commercial use (6,695 litres of red, 7,737 litres of white, 3,427 litres of rosé) from a vineyard area of 20 hectares. Current figures are likely to be at least double these.

The recent warming of Europe has made a big difference, of course, as it has in other areas (see the recent post on Fifty years of Bordeaux vintages). This topic is discussed in more detail elsewhere: Sweden's wine industry is maturing nicely — thanks to climate change (and see also: The feasibility of Ireland becoming a wine producing country due to climate change; and Netherlands and Belgium to gain cross-border PDO). In addition, proximity to the sea helps moderate the climate in most of Sweden's vineyard areas (the Baltic Sea is quite shallow and therefore relatively warm).

And, yes, you have never heard of most of the vine cultivars that are grown. For example, many are particularly hardy hybrids developed in Germany (eg. Solaris) or Czechoslovakia (eg. Rondo). And, no, I don't think that the resulting wines will take Australia by storm, or anywhere else for that matter.

More than a century of Barolo vintages — Marchesi di Barolo

In the northern Italian vineyard region of Piemonte, two well-known Barolo producers have provided publicly available comments on 100 years of their wine vintages. Finding these online is not easy, and so I will be covering them in this blog. I have already presented A century of Barolo vintages — Fontanafredda.

Here, I have listed the chart from Marchesi di Barolo (formally: Cantine del Marchesi di Barolo S.p.A.). You can read all about the estate and winery at the Marchesi di Barolo web site (in several languages).

I obtained this listing from the 心無罣礙 – 抱青 website, which in turn got it from a page (that no longer exists) at the Marchesi di Barolo site. There are apparent errors in the source, which I have corrected here. I have provided a brief analysis of the data at the end of the post.

A Century of Barolo Vintages 1861-2005

The 145 vintages are scored on an increasing scale:
    Poor, Average, Good, Excellent, Great, Exceptional.
However, the first two categories are not actually distinguished in the list; and comments on the vintage are provided only for the latter four categories.

Year	Harvest quality	Comments
1861	Excellent	A limited harvest. Wine of great structure and balance with intense perfumes and pronounced flavor. A wine for long ageing.
1862	Good	A full-bodied wine with pronounced tannins and acidity. Aromas are fairly intense and complex.
1863
1864
1865	Excellent	Meagre harvest due to severe spring frosts. The grapes that were harvested, however, were very healthy and mature, creating wines of good structure and pedigree.
1866
1867	Good	Normal level of production with healthy, mature grapes. The resulting wines are medium-bodied, robust and harmonious. Some retained a touch of sweetness.
1868	Exceptional	An extremely well-structured wine with remarkable perfumes that are both very intense and very appealing. A wine for long cellaring.
1869	Good	A wine which seems weak at first but reveals a ray of ample, pleasant aromas. The palate sensations are at once delicate and decisive. For these reasons, this wine could be considered one for easy drinking.
1870	Good	Somewhat lacking in structure but nonetheless very pleasant to drink. Firm, supple tannins guarantee a moderate capacity for ageing.
1871
1872
1873	Great	A wine of great structure and vigor. Extraordinary perfumes and pronounced tannins render it suitable for long ageing.
1874
1875
1876	Good	A tremendously structured wine with firm tannins that initially seem very hard. Extremely intense and persistent perfumes.
1877
1878
1879	Excellent	An abundant harvest of healthy, mature grapes produced wines with excellent structure, soft tannins and good acidity. Excellent prospects for a long period of maturation.
1880
1881
1882
1883
1884
1885	Good	A very sturdy, pleasant and inviting wine, even if a bit light in body and structure.
1886	Great	Scarce harvest due to heavy hailstorms during the summer. However, the grapes that were harvested were beautiful, ripe and perfectly healthy. The wines are very robust and well-structured, making them good candidates for extended ageing.
1887	Excellent	A wine of great robustness and structure. Perfumes are very pronounced, intense and appealing. A vintage to remember.
1888
1889	Excellent	A wine with tremendous body and character. Aromas are intense and persistent with incredible structure and equilibrium. Firm tannins, which at first come off as soft and unobtrusive, and good acidity are clear indications of its promising future.
1890	Good	Though this excellently structured wine initially seemed a bit coarse and rough around the edges, over time it has developed a good equilibrium with intense and pleasing aromas. This is not, however, a wine that will benefit much from long ageing.
1891
1892
1893
1894	Excellent	Extremely ripe, healthy grapes produced a wine that is well-structured and high in alcohol yet perfectly balanced. Very intense, persistent aromas. Drinking well in its youth, it also shows excellent potential for long ageing.
1895	Good	Rich and well-structured, with intense and moderately long perfumes. Good balance between tannin and acidity. A wine which is both harmonious and pleasant to drink.
1896
1897	Exceptional	The grapes, which were actually very scarce due to strong spring frosts, matured perfectly during the subsequent growing season and were harvested in perfect condition. The result is a wine with tremendous structure and powerful alcohol. Very intense and persistent aromas. A perfect candidate for long ageing.
1898	Excellent	Rich and well-structured with intense, moderately long perfumes. Good balance between tannin and acidity. A wine which is both harmonious and pleasant to drink young, but capable of ageing as well.
1899
1900
1901
1902
1903	Good	A wine with a big structure, initially slightly unbalanced but developing fine equilibrium as it matured.
1904
1905	Good	An imposing wine because of its impressive elegance, structure and rich sensations; generous and promising.
1906	Good	Well-structured wine, abounding in tannins and acids; strong, forthright sensations.
1907	Great	A wine of great breeding and superb balance. Famous for its generosity and austere elegance that persisted for many years.
1908	Excellent	A wine with a good structure but not richly endowed with color or tannins. Impact on nose and palate somewhat weak but pleasant.
1909
1910	Excellent	Austere, rich wine with good impact and a fine balance among its characteristics.
1911	Good	A wine with a great structure, well-endowed with tannins and acids. Pronounced disequilibrium initially but unusually promising.
1912	Great	An imposing, generous and robust wine of fine breeding. Surprising because of its austere balance and rich sensations.
1913
1914
1915
1916
1917	Excellent	A wine with a great personality. Full and intense aromas and flavors distinguished by their austere generosity and elegance.
1918
1919	Great	A wine of superb breeding. Rich and persistent aromas; flavors that were imposing because of their intensity, balance and harmony.
1920	Good	A generous, well-structured wine with intense and agreeable sensations.
1921
1922	Exceptional	An opulent wine with remarkable aromas and flavors that melded in a triumphant blend.
1923
1924	Good	A well-structured, hardy wine with pleasant although not intense or persistent aromas; forthright tannins that, in time, developed balance and harmony.
1925	Excellent	A rich, unusually refined wine because of its complex, intense and persistent aroma and exhilarating flavor, harmoniously combined.
1926
1927	Excellent	A wine whose strength and generosity were well expressed in its range of aromas and flavors.
1928	Great	A great wine in all its characteristics, with a fine concentration of sensations. The harvest was weak after a long and frigid winter and disastrous hailstorms.
1929	Exceptional	Extraordinary wine with a complete range of intense, full and persistent odors and possessing a wealth of flavor sensations of surprising intensity. The wine showed remarkable balance and harmony.
1930
1931	Exceptional	Extraordinary wine with a complete range of intense, full and persistent odors and possessing a wealth of flavor sensations of surprising intensity. The wine showed remarkable balance and harmony.
1932
1933
1934	Excellent	A generous, well-structured wine with forthright tannins and acids; characteristic and distinctive aromas and flavors in a complex equilibrium of notable austerity.
1935
1936
1937	Good	Generous, well-structured wine with good and persistent odors and pleasant and relatively complex flavor sensations.
1938
1939
1940
1941
1942
1943
1944
1945	Good	A robust, well-structured wine, distinctive for its intense and pleasant aromas and properly balanced flavors of fine complexity.
1946
1947	Exceptional	The wine possessed all the elements of greatness: aromas of rare intensity and appeal; noble, unusually persistent flavors; balanced and harmonious sensations of incomparable softness.
1948
1949
1950
1951	Good	A well-structured wine, richly blessed with pleasant and intense odors and complex, attractive and persistent flavors; exceptionally well-balanced and harmonious. Production was limited because of violent hailstorms that ravaged many vineyards.
1952
1953
1954
1955	Good	Remarkable for its structure and generosity, the wine offered intense aromas with good persistence and complex, attractive flavors; notable equilibrium and harmony.
1956
1957	Excellent	A wine with a big, full and generous structure. A wealth of intense and persistent aromas; remarkable equilibrium and harmony that contributed to a velvety sensation.
1958	Great	A warm, full-bodied and well-structured wine. Intense, remarkably full aromas; a desirable touch of acidity and a substantial supply of sweet tannins combined to assure softness and elegance.
1959
1960
1961	Great	A wine of superb class characterized by a substantial structure and aromas of great depth and breadth. Sweet tannins and moderate acidity yielded a full and harmonious wine of exceptional elegance.
1962
1963
1964	Great	A majestic wine because of its structure and complexity. Noble aromas that were full and persistent, forward tannins that were assertive but, in combination with all other elements, they assured an unusually long life.
1965
1966
1967	Excellent	A wine with outstanding structure and body, a pleasant and intense aroma, tannins, tending toward sweet, that were evident but of good quality and pronounced sapidity, characteristics that promised a brilliant future.
1968	Good	A wine with little structure but endowed with appealing and intense aromas. Rich flavor sensations tending toward seductive softness.
1969
1970	Great	A wine blessed with a magnificent structure, generous and richly endowed. Its strength was such that it could accept a substantial content of tannins and acids that, supported by the warmth of the alcohol, assured remarkable sensations of equilibrium and harmony.
1971	Exceptional	In addition to its attributes of fine breeding, imposing structure and rich personality, which were also found in the 1970 Barolo, this wine offered an impressive wealth of aromatic sensations and a flavor that was exceptionally well-balanced and notably elegant.
1972		[Such a poor year for the nebiolo grapes that it was voluntarily declassified by all Barolo and Barbaresco producers.]
1973
1974	Excellent	A wine of good breeding with a rich body and tannins and acids that were forward but well balanced. A full, intense aroma that was extremely pleasant, inviting and persistent; flavor sensations marked by an austere elegance.
1975	Good	A rich, well-structured wine with intense and relatively persistent odors, a good balance between tannins and acids; a harmonious, drinkable wine.
1976
1977
1978	Great	A well-structured wine, with a rich and vigorous body. Intense aromas of outstanding fullness and persistence; excellent equilibrium and harmony among all elements. The tannins were forward but tending toward the sweet. A modulated, elegant wine.
1979	Excellent	This rich and complex wine showed fine breeding. Elegant, extremely intense and well-modulated aromas; substantial tannins and acids that assured an austere and balanced wine.
1980	Excellent	A robust, well-structured wine. Full and encompassing aromas; outstanding balance and an exhilarating, unusually harmonious finish.
1981
1982	Exceptional	A rich wine with a big and complex structure. Noble aromas that were full and persistent in perfect harmony with complex, velvety and elegant flavors. Undoubtedly a vintage of outstanding longevity.
1983
1984	Good	A complex wine although lacking in structure. Delicate but intriguing and persistent aromas; outstanding flavor sensations but somewhat hard due to excessive tannins and acidity. Apparently disharmonious but promising. Weak production caused by a late frost.
1985	Great	A wine with an extraordinary structure and complexity. A wealth of elegant and enveloping aromas. Sweet tannins and balanced acidity combined in flavors that were soft and elegant.
1986	Good	This wine had a big structure and outstanding complexity but yield was reduced because of a fierce hailstorm in the spring. Complex and forthright aromas; imposing but evolved tannins with a flavor in perfect balance with the big body.
1987
1988	Excellent	A well-structured wine with remarkable richness of components. Intense, full and pronounced aromas and a full, perfectly balanced flavor despite extremely forward tannins and acids. A promising wine with the certainty of a long life.
1989	Great	An impressive wine because of its big and generous structure. Extremely intense and remarkably full and persistent aromas of pronounced sapidity. Holding great promise for the future.
1990	Exceptional	Because of its extraordinary attributes, the wine displayed a majestic opulence. Aromas of an enormous range of full and elegant sensations. The flavor showed great harmony and balance against a remarkably velvety background.
1991	Good	Despite its apparent delicacy, this wine featured a superb range of full and appealing aromas seconded by delicate and impressive flavor sensations. Because of those qualities, it was regarded as a highly drinkable wine.
1992
1993	Excellent	A wine with a good structure and a commendable balance among its components. Intense and well-expressed aromas and relatively forward tannins but with good prospects for further development. Remarkably harmonious.
1994
1995	Excellent	A wine with great character and big body, the fruit of the few grapes saved from a ferocious hailstorm that battered the vines during the summer. Intense and pronounced aromas; structured and balanced flavors; noble tannins and good acidity that offer excellent promise for the future.
1996	Great	A wine that is imposing and noble in character and structure. Its intense, full and persistent aromas are in perfect harmony with a rich array of visual and flavor sensations. Outstanding vinosity, proper acidity and forthright tannins that are steadily softening give this wine a majestic austerity that assures a long life and remarkable elegance.
1997	Exceptional	An extraordinary wine! The richness of all its components is as noteworthy as it is rare. Possessing great strength because of its alcohol level and striking in appearance because of its intense, deep ruby-red color, it is certain to please even the most demanding connoisseurs. Its complex, full and persistent bouquet is highly provocative, while its noble, balanced, harmonious and full-bodied flavors, which are without equal, are accompanied by an unusually persistent aromatic intensity.
1998	Great	Noble and strong wine. Its colour and intensity are just perfect. It has an intense, rich and persistent bouquet with scents of ripe fruit, sweet spices and undergrowth. Moderate acidity and marked but sweet tannins confer good balance and ease drinking. It is a solid promise for the years to come.
1999	Exceptional	This vintage wine has extraordinary characteristics. It is gifted by nature with all the possible elements that express themselves generously. Its bouquet is infinitely rich with elegant and persistent notes. Extraordinary balance and harmony livened by a remarkable velvety sensation. Long life and big promises.
2000	Great	This wine has wonderful looks. The beautiful shade and intensity of its colour and the garnet nuances confirm a correct evolution. Its rich, full and persistent bouquet is highlighted by notes of vanilla and spices, as a result of its noble lineage and maturation in precious wood. It feels warm, sober, rich and well structured in the mouth. The balance of acids and tannins reveals excellent chances of evolution. The finish is long with a pleasing aromatic persistence.
2001	Exceptional	Garnet-red with deep ruby reflections exalted by a remarkable transparency. Copious ‘tears’ confirm its generous austerity. Exotic aromas of dried flowers, small fruits and wood essence combine to form a bouquet of extraordinary complexity. On the palate one is struck by heat, power, structure and tremendous equilibrium. Alcohol, acidity and tannin co-exist with amazing harmony and persistence.
2002	Good	Brilliant garnet-red color that, if not very intense, is still quite alive and promising. The bouquet is full and persistent with subtle hints of vegetable, flower and spice. There is the light secondary aroma of vanilla from the wood in which the wine matured. The palate impression is decidedly dry, warm and harmonious. Moderate acidity is balanced by the sensation of soft, supple tannins. Overall, pleasant and well-structured.
2003	Excellent	A vintage which will be remembered for its prolonged drought and extreme heat. Optimal in visual appearance and rich in body, with pronounced though perfectly balanced acids and tannins. Full and persistent perfume of balsamic vinegar with secondary aromas of ripe fruit and mountain hay. The palate is dry, warm and engaging. An austere, important wine with a long, lingering aromatic finish.
2004	Great	Brilliant wine with a lively garnet-red color that is both intense and appealing. The bouquet is intense, rich, full and complex. It opens up slowly with evident notes of sweet spices, licorice, berry fruits and dried flowers. There is a subtle hint of vanilla in an otherwise decisive, forward wine. On the palate, this wine is distinguished by its elegant structure, pleasant sapidity and soft, almost sweet, tannins. A true ‘purebred’ which will surely continue to evolve and develop for a long time to come.
2005	Great	Beautiful, intensely brilliant garnet-red color. Impressively rich bouquet recalls the fragrance of roses, sweet spices and fresh inviting fruit. On the palate there is a magnificent balance between the power of its alcohol, its firm tannins (which are already showing a tendency to soften around the edges), and a pleasantly vibrant sapidity. A wine of great breeding and nobility with a long, lingering aromatic finish.

Analysis of the data

Summary of the harvest quality data for the 145 vintages:

Exceptional	12
Great	18
Excellent	22
Good	24
Average/Poor	68
Declassified	1

The declassified vintage in 1972 is discussed in the post A century of Barolo vintages  — Fontanafredda.

The harvest quality data here cannot be shown as a time series, as they were for the Fontanafredda vintages, because the Poor and Average data have not been included. However, the vintages for which there are available data (55 of the vintages from 1906 onwards) can still be compared to those for Fontanafredda. This comparison is shown in the table below.

As expected for two producers in the same region, there is a close association between these two datasets. The vintage quality was scored identically in 33 cases (60%; marked in orange in the table), and differed by only one category in a further 19 cases (35%; marked in bright yellow). However, the 1928 vintage was scored as Normal by Fontanafredda but Great by Marchesi di Barolo, which is a difference of three categories.

During the 80 years from 1906 onwards, there were only four occasions when both producers rated the vintage as best: 1922, 1931, 1947 and 1971. However, during the next 20 years there were another four occasions: 1990, 1997, 1999 and 2001. This emphasizes just how different European grape harvests have been since 1990. This is discussed in more detail in the post Two centuries of Bordeaux vintages.

Biases in wine quality scores

People often consult wine critics when they want to know how various wines compare, and these days those critics often communicate this information using quantitative rating scales (see Wine by the numbers).

While contemplating the meaning of the numerical scores handed out by wine critics (What's in a number? Part the second), Alex Hunt produced various graphs showing the frequency distribution (see Wikipedia for an explanation) of the marks awarded by a selection of those critics. This selection included Jancis Robinson, Robert Parker and James Halliday.

For these people, Hunt pointed out various features that make each graph unique — this reflects the fact that these critics all use the wine scores in individual ways. This is an important point, because it seems to be widely thought that "a wine score is a wine score"; but this is not so. Each critic has their own individual way of interpreting, and thus using, wine scores.

Frequency distributions

To do his analysis for each person, Hunt downloaded (from various websites) a collection of the scores they had assigned to the wines they had evaluated, on whatever points-scale system they had used (usually the 20-point or 100-point scale). From this, he constructed a set of graphs showing how often each person had awarded each score. As an example, this first graph is the one he constructed for Robert Parker, based on "43,094 scores from the online edition of The Wine Advocate, spanning the last four years." The height of each vertical bar in the graph represents the proportion of wines receiving the score indicated.

Hunt's stated purpose was to show:

that distributions are wholly personal, and can vary quite considerably between individuals who, on the face of it, use the same scoring system. There will [also] be evidence of ... the idea of 'significant scores' — that some numbers have more salience attached to them, and harbour an unexpectedly large number of wines within their boundaries, at the expense of their immediate neighbours.

So, Hunt is suggesting that wine assessors often have subconscious biases about what scores to assign, and that their particular biases become obvious when you plot a frequency distribution of lots and lots of their wine scores. It is this point that is of interest in my blog post — that some critics show individual biases in the scores that they award.

For the Parker graph, Hunt notes two biases:

A significant spike appears at 90, and 89 is unusually low — contrary to expectation, lower than 88. Could it be that a certain proportion of wines are just tickled over the finish line into the most salient category of wine scores in the world: 90+ RP? [wines with scores of 90 Parker points or more] ... The effect is repeated for that other super-salient category: 100 RP ... here, we see a dip at 99, then a marked increase at 100. Should we conclude that there are more perfect than nearly perfect wines? No; just that the category of 100-pointers has been allowed to swell rather.

That is, some of the "90" scores should actually have been 89, in an unbiased world, and that some of the "100" scores should actually have been 99. It would be interesting to estimate just how many scores are involved, as this would quantify the magnitude of these two biases.

Estimating the biases

However, I do not have the Parker data available, and so I will illustrate this point with a different dataset. This dataset concerns 335 Sicilian wines reviewed in the article Sicily's Star Rises, by Alison Napjus in the Wine Spectator magazine. These wines were scored on the 100-point scale; and the frequency distribution of those scores is shown in the next graph.

For these data, the "expectation" that Hunt refers to (in the quote above) is that, in an unbiased world, the point scores would show a relatively smooth frequency distribution, rather than having dips and spikes in the frequency at certain score values (such as 90 or 100). In the data for the Sicilian wines, it seems that there are two unexpected spikes or dips: (i) a score of 87 is less frequent than a score of 86; and (ii) a score of 90 is more frequent than a score of 89. These both contradict the expectation of a smooth distribution.

Thus, it seems that the scorer(s) preferred even-numbered scores to odd-numbered scores. This idea can be investigated by working out what the "expected" values would be, in an unbiased world.

Mathematically, the expected scores would come from an "expected frequency distribution", also known as a probability distribution (see Wikipedia). There are many such theoretical distributions that have been devised by mathematicians, for many different purposes. There is a simple model behind each of these distributions, and all we have to do is choose a model that seems appropriate for the type of data that we have — this then tells us which probability distribution to use.

It seems to me that a Weibull distribution (see Wikipedia) would be suitable for wine-score data (for both the 20-point and 100-point scales). One way to think about this distribution is that it models the maximum values of a series of observations that have an upper limit. What this means in our case is: (1) the assessors assign to each wine the highest score that they believe it deserves ("this wine deserves no more than an 89"); and (2) the scores are much closer to the upper limit of 100 than to the lower limit of 50. Both of these ideas seem reasonable.

We can fit this model to the data using standard mathematical techniques (involving the principle called maximum likelihood), which will then provide the expected frequency of each score. In order to work out the degree of bias in the observed scores of 89 and 90 (too many scores of 90 and not enough 89s), we then need to repeat this analysis, each time changing the observed frequencies of both 89 and 90.

For example, the Wine Spectator data have 31 wines with a score of 89 and 55 wines with a score of 90. So, we repeat the fitting of the Weibull distribution with 32 scores of 89 and 54 scores of 90, instead, and then do it again with 33 scores of 89 and 53 scores of 90, and so on. Eventually, we will find out which frequencies of 89 and 90 are optimal for our model; and these will be our estimates of what would happen in an unbiased world (the "expected" values).

The resulting frequency distribution is shown in the graph immediately above. Note that it is a nice smooth distribution, with no dips or spikes.

This expected distribution is compared directly with the observed distribution in the final graph. In this graph, the blue bars represent the (possibly biased) scores from the Wine Spectator, and the maroon bars are the unbiased expectations (from the model). Those scores where the heights of the paired bars differ greatly are the ones where bias is being suggested.

Note that this comparison shows expected frequencies for scores of 87, 89 and 90 that are very different from those produced by the Wine Spectator, implying that all three of these scores show bias. (As an aside, scores of 83 and 93 also seem to show a bit of bias.) So, the final result is that c. 18 of the scores of "90" would have been 89 in an unbiased world. That is, 32% of the scores of 90 show bias!

A score of 90 is, indeed, much more important for some wine assessors than is a score of 89, exactly as suggested by Alex Hunt (when studying Robert Parker's scores).

This same type of analysis could be applied to any other wine scorers. Indeed, if you were a mathematician you might also like to try other models, to see whether it makes much of a difference.