Much literature shows that the ratings assigned by wine judges are uncertain, some authors have proposed that judges be tested, and a few wine competitions do test judges. However, no literature or competition has yet proposed a test or rating for judges based on realistic competition conditions. This article uses coefficients of multiple correlation to rate each of 54 judges who assigned ratings to 2,811 wines entered in a commercial competition. Results show that there is a strong and positive correlation between the ratings assigned by most judges to most wines. However, those correlations also show that the ratings assigned by approximately 10% of judges are indistinguishable from random assignments. Using correlations to rate the raters, a program is underway to monitor those judges and variations in competition protocol that may affect their ratings. (JEL Classifications: A10, C00, C10, C12, D12)

Evaluating observed wine-tasting results as a mixture distribution, using linear regression on a transformation of observed results, has been described in the wine-tasting literature. This article advances the use of mixture models by considering that existing work, examining five analyses of ranking and mixture model applications to non-wine food tastings and then deriving a mixture model with specific application to observed wine-tasting results. The mixture model is specified with Plackett-Luce probability mass functions, solved with the expectation maximization algorithm that is standard in the literature, tested on a hypothetical set of wine ranks, tested with a random-ranking Monte Carlo simulation, and then employed to evaluate the results of a blind tasting of Pinot Gris by experienced tasters. The test on a hypothetical set of wine ranks shows that a mixture model is an accurate predictor of observed rank densities. The Monte Carlo simulation yields confirmatory results and an estimate of potential Type I errors (the probability that tasters appear to agree although ranks are actually random). Application of the mixture model to the tasting of Pinot Gris, with over a 95% level of confidence based on the likelihood ratio and t statistics, shows that agreement among tasters exceeds the random expectation of illusory agreement. (JEL Classifications: A10, C10, C00, C12, D12)

Results for a total of 804 double-blind tastes by experienced tasters during nine tasting events are reported. T-test results reject the hypothesis that flight-position bias affects results. The distribution of ranks for a wine is a mixture distribution, and tests concerning the variance of that mixture distribution do not isolate the variance due to the randomness mixture component alone. T-statistics for the mean ranks of high- and low-ranking wines are over several standard deviations from a random expectation. T-tests show that the statistical significance of the difference between wine ranks is positively related to the difference in their mean ranks. At a 95% level of significance, the difference in ranks between the first- and second-place wines appears to be significant in 33% of tastings. At 95%, the difference in ranks between the first- and last-place wines appears to be significant in 100% of tastings. Monte Carlo simulation shows that much of those differences could be illusory and due to ranking procedures that lead to Type I errors. While the mean correlation coefficient between price per bottle and mean preference is a weakly positive 0.23, this may not indicate an inefficient market. (JEL Classifications: A10, C00, C12, D12)

The inability of many wine judges to achieve perfect consistency by assigning the same rating to the same wine in a blind tasting is well established. Results for four wine tastings that include blind replicates are examined in this article. Although perfection is rare, the probability distributions of those results show that wine judges do tend to assign closer ratings to replicates than is likely due to chance alone. Approximately one-third of judges assign ratings that are within one rank of perfect consistency, and two-thirds assign ratings within two ranks of perfect consistency. This finding is sensitive to judges’ capabilities, the mechanics of the tasting protocol, and the extent to which the replicate is different from other wines in the tasting. Much wine-related research to date takes judges’ individual ratings as deterministic, yet these results show that those ratings are stochastic. These results yield a probability distribution that may guide future research concerning the uses and economic implications of wine ratings. (JEL Classifications: A10, C10, C00, C12, D12)

More than forty published works show that women and men differ in their taste preferences for sweet, salt, sour, bitter, fruit, and other flavors. Despite those differences, dozens of state fair and other wine competitions determine winners' ribbons, medals, scores, and ranks by pooling the opinions of female and male judges. This article examines twenty-three blind wine tastings during which female and male judges scored more than nine hundred wines. Two-sample t-test results show that the gender-specific distributions of scores do have similar means and standard deviations. Exact p-values for two-sample chi-square tests show that the distributions of men's and women's scores are not significantly different, and exact p-values for likelihood ratio tests of Plackett-Luce model results show that the genders' preference orders are not significantly different. The correlation coefficient between women's and men's scores is weakly positive in 90 percent of the tastings. On that evidence, indications that the genders prefer different wines are difficult to detect. If such differences do exist, as the nonwine literature implies, the results of this analysis show that those differences are small compared to non-gender-related idiosyncratic differences between individuals and random expressions of preference. The potential for accept-a-false-null-hypothesis Type II error when pooling female and male judges' wine-related opinions appears to be small. (JEL Classifications: A10, C10, C00, C12, D12)