The role of cross-validation approaches in sound quality metric development

Abstract

The sound of a product is an important feature influencing the perceived quality impression. Thus, product sound is deliberately designed to stand out against competitors. To determine meaningful design targets guaranteeing a perceived sound quality, jury tests are performed and sound quality metrics are developed. Ideally, by means of a developed sound quality metric, the prediction of perceived sound quality of comparable products is possible without the need for further jury testing. However, several aspects influence the reliability and validity of jurors’ ratings deteriorating the prediction accuracy of the derived sound quality metric. Moreover, if parameters of a sound quality metric are only estimated in a way to give a best fit with the rating data, the external validity of the metric is possibly reduced. In order to estimate the robustness of a sound quality metric and to avoid overfitting, cross-validation methods and different error measures should be considered. The paper will illustrate the impact of missing robustness analyses on the validity of sound quality metrics and how systematic approaches can be applied to improve the process of metric development. Moreover, general limitations and risks of sound quality metrics based on jury tests will be discussed.