Reliability of categorical versus continuous scoring of welfare indicators: lameness in cows as a case study

Abstract

Many animal welfare traits vary on a continuous scale but are commonly scored using an ordinal scale with few categories. The rationale behind this practice is rarely stated but appears largely based on the debatable conviction that it increases data reliability. Using 54 observers of varying levels of expertise, inter-observer reliability (IOR) and user-satisfaction were compared between a 3-point ordinal scale (OS) and a continuous modified visual analogue scale with multiple anchors (VAS) for scoring lameness in dairy cattle from video. IOR was significantly better for the VAS than for the OS. IOR increased with self-reported level of expertise for the VAS, whereas for the OS it was highest for observers with a moderate level of expertise. The mean continuous scores and the mean categorical scores were highly correlated. Three times as many observers stated a preference for the VAS (n = 27) compared to the OS (n = 9) in investigating differences in lameness between herds. Contrary to common perception, these results illustrate that it is possible for a continuous cattle lameness score to be more reliable and to have greater user acceptability than a simple categorical scale. As continuous scales are also potentially more sensitive, and produce data more amenable to algebraic processing and more powerful parametric analyses, the scepticism against their application for assessing animal welfare traits should be reconsidered.

Ordinal or visual analogue scales for assessing aspects of broiler chicken welfare?

“…This also implies that data from different studies can be compared if the same ORS are used. However, assessing continuous welfare traits by using discontinuous scales may be disadvantageous (Tuyttens, Sprenger, Van Nuffel, Maertens, & Van Dongen, 2009). The use of ORS may result in reduced sensitivity when raters are able to discriminate more levels of the assessed indicator than the number of categories allow for and are forced to group gradations they perceive as different into the same category.…”

“…A different type of scale, the visual analogue scale (VAS), is largely used to assess pain in humans and non-human animals ( de Grauw & van Loon, 2016;Hjermstad et al, 2011). In AW assessment, VAS has also been applied to assess qualitative behavior defined by Wemelsfelder, Hunter, Mendl, and Lawrence (2001) as one of the "whole animal" measures that aim to assess the overall subjective experience or mood of an animal (Fleming et al, 2016;Grosso et al, 2016;Minero et al, 2016), and lameness (Flower & Weary, 2006;Nalon et al, 2014;Tuyttens et al, 2009;Vieira, Oliveira, Nunes, & Stilwell, 2015) in different species. VAS is a continuous scoring system that consists of a line, which varies usually from 100 to 125 mm in length, anchored by the minimum and the maximum score at each end.…”

“…The downside of the conventional VAS is the difficulty to train raters to score different gradations consistently, and as observed by de Grauw and van Loon (2016), the inter-rater reliability may be low. In this case, the tagged VAS (tVAS), which is a VAS with internal anchors, has been investigated as a tool to combine the advantages of both ORS and VAS (Nalon et al, 2014;Tuyttens et al, 2009). The tags add information to guide raters through different gradations thereby increasing reliability and facilitating the training of raters (Tuyttens et al, 2009).…”

“…In this case, the tagged VAS (tVAS), which is a VAS with internal anchors, has been investigated as a tool to combine the advantages of both ORS and VAS (Nalon et al, 2014;Tuyttens et al, 2009). The tags add information to guide raters through different gradations thereby increasing reliability and facilitating the training of raters (Tuyttens et al, 2009).…”

“…Previous studies assumed equidistant tags to VAS to assess specific indicators of animal welfare based on existing categories used in ORS (Meeremans et al, 2017;Nalon et al, 2014;Rufener, Baur, Stratmann, & Toscano, 2018;Tuyttens et al, 2009). However, Vieira et al (2015) challenged this rationale by presenting a non-equidistant characteristic of tags in VAS as with lameness in dairy goats.…”