Judging reliability at wine and water competitions

Abstract

Studies suggest the inter-rater reliability of judges at wine competitions is higher than what would be expected by random chance, but lower than what is observed when experts in other fields make judgments specific to their expertise. To further contextualize the (un-) reliability of wine judging while also extending the study of fine water, we examine the inter-rater reliability of judges at an annual international competition for bottled waters. We find that the inter-rater reliability of water judging is generally better than chance and, at best, about the same as the inter-rater reliability of wine judging at some wine competitions. These results suggest that perceptible differences between fine waters exist but are less pronounced than those between fine wines and, also, that aesthetic standards with respect to fine waters exist but are currently less established than those for fine wines.

Multifactor and multidimensional data quality analysis of judge scoring in diving competition

“…In sports competitions, judge scoring data serve as an objective measure of an athlete’s performance level. However, research has indicated the unreliability of objective measurements ( Bodington, 2017 ; Martire and Montgomery-Farrer, 2020 ; Berg et al, 2022 ). Controversy often arises regarding the quality of judge scoring data, undermining fairness and justice in sports competitions ( Looney, 2004 ).…”