The 2016 Wines of Portugal Challenge: general implications of more than 8400 wine-score observations

Abstract

The Wines of Portugal Challenge is an annual competition among wines produced by over 1000 vintners in over 30 of the country's wine growing regions. In 2016, judges assigned scores to over 1300 wines resulting in over 8400 wine-score observations. Analysis of that large sample yields implications about wine judges' ratings that are difficult to detect with statistical significance in the small samples that are typical of most wine tastings. The Challenge's frequency distribution of scores showed left skewness and local peaks just below the score thresholds for bronze, silver and gold awards. Student's t-tests showed that there were no significant differences in scores assigned by gender-of-judge, nationality-of-judge and to wines from different regions. However, judges did assign higher scores to sweet wines than to other types of wine. While the dispersion in scores was material, p-values showed that the aggregate order of rating was very unlikely to be random and the distributions of mean scores showed that the strengths of judges' preferences against the least-preferred wines were stronger than those in favor of the most-preferred wines. Ties between wines' mean scores were common and could be broken by several methods including the preference probabilities implied by a Plackett-Luce model.

Polyphenol Composition by HPLC-DAD-(ESI-)MS/MS and Bioactivities of Extracts from Grape Agri-Food Wastes

“…Europe is recognised for its high-quality wines, and in this scenario, Portugal stands out for its traditional and expressive production of wines with Protected Designation of Origin (PDO) and Geographical Indication (PGI) labels, highlighting the distinctive wine production across the 14 different regions of the country [3,4]. To exemplify it, in a recent competition in Portugal, over 1300 Portuguese wines were sensorially evaluated by 151 experts and most received scores ranging between 80 and 90 on a 0-100 scale, corroborating the level of quality and diversity of the wines produced [5].…”

Stochastic error and biases remain in blind wine ratings

“…For example, the International Organization of Vine and Wine (OIV, 2021) prescribes scoring between 0 and 100 along with thresholds for Bronze, Silver, and Gold medals at scores of 80, 85, and 90, respectively. For 8,400 ratings according to the OIV system, Bodington and Malfeito-Ferreira (2017) showed spikes in the frequencies of scores assigned just below those thresholds.…”

A maximum entropy estimate of uncertainty about a wine rating

“…For example, the OIV system sets score thresholds for Bronze, Silver, and Gold medals at scores of 80, 85, and 90, respectively. 2 With a sample of 8,400 ratings, Bodington and Malfeito-Ferreira (2017) showed spikes in the frequencies of scores assigned just below those thresholds. Thus, some judges appear to anchor at OIV's category thresholds.…”