Study of consistency of expert evaluations of wine sensory characteristics by positional analysis

Abstract

Individual characteristics inherent in the expert, as well as their physical and psycho-emotional state subject to the influence of random, uncontrollable factors, contribute to subjectivity in the sensory evaluation of wines. With great variability of opinions, the final results of sensory evaluation may become doubtful. The presence of a random component in the sensory evaluation justifies the use of statistical methods for analyzing the consistency of expert evaluations. Along with Spearman's correlation coefficients and Kendall matching, Cronbach alpha criterion was used to assess the consistency of expert opinions. The advantages of positional analysis have been discussed -Cronbach's alpha criterion is calculated not by the rank of expert points, but by the initial point scale considering its variability; it allows to evaluate the contribution of each expert to the consistency of expert evaluations, as well as the reliability of the total scale of points set for each wine sample. Based on the data analysis from sensory evaluation of the quality of dry red and white wines of Russian production, the results of the consistency study of expert evaluations as well as the reliability of the total score scale have been presented. What is more, analysis of the "loyalty" of experts in evaluating the quality of wines has been performed.

How to Improve the Monitoring of Subjective Analytical Methods and Assessors - The Case of Wine

“…Most often, papers deal with methods and analyses of the quality of sensory testing in research and food industry [12,13]. The competencies and the results of assessors on International wine competitions are also the subject of interest [14][15][16][17]. However, evaluating the sensory compliance of food products and wine with Protected Designation of Origin (PDO) with the sensory description in official specifications is difficult because there are no standard methods for such testing [18].…”

“…Pinto [27] recommended pre-testing panellists with the Triangle test and Cronbach's Alpha coefficient as a measure of assessor's consistency. Some other authors [17] also presented Cronbach's Alpha criterion for positional analysis because it allows a deeper look at the consistency and variability of assessors. This model could be useful when assessors rarely participate in testing or are limited to a specific product or parameter.…”

Sensory Profile of 3 in 1 Instant Coffee Using Emotional-Sensory Mapping, Flash Profile, and Cata (Check-All-That-Apply) Methods

“…Penalty analysis on the CATA method can only be carried out if data on preferences and ideal product data were available. This analysis is divided into five groups of sensory attributes, namely must have, nice to have, does not influence, does not harm, and must not have (Khalafyan et al, 2021;Meyners et al, 2013), but only must-have, nice to have, and must not have. which will be analyzed because it has more influence on product development.…”