Implementing Welfare Quality® in UK assurance schemes: evaluating the challenges

Abstract

This paper presents an account of a Welfare Quality® assessment of 92 dairy farms carried out by seven experienced assessors. The aim was to evaluate the potential of the Welfare Quality® assessment protocol with respect to its uptake by UK farm assurance schemes. Data collection, and measure aggregation were performed according to the Welfare Quality® protocol for dairy cows. This study examined the data itself, by the testing of how hypothetical interventions might be reflected in changes in the aggregated scores, and also investigated human-related aspects, through inter-assessor standardisation sessions to evaluate reliability, and an assessor focus group to collect feedback. Overall, three main ‘challenges’ were identified. The first challenge related to the large amount of missing data. Unexpectedly, this was such that it was only possible to calculate an overall classification for 7% of farms. The second challenge concerned the way in which aggregated scores did not always reflect hypothetical interventions. The final challenge was inter-assessor reliability, where not all assessors were found to achieve acceptable levels of agreement on a number of outcome measures by the third training session. Suggestions for managing these challenges included, follow-up to assessor training, the use of multiple imputation methods to fill in missing data, and, where applicable, not aggregating the scores. The conclusion of the study was that the protocol provided useful information from which to make an informed selection of measures, but that the challenges, combined with the lengthy assessment time, were too great for its use as a certification tool.

Navigating the iceberg: reducing the number of parameters within the Welfare Quality® assessment protocol for dairy cows

“…The collection of the data used in this paper, along with a list of the measures and any missing values associated with those measures, has been previously described in Heath et al (2014). Briefly, full Welfare Quality ® assessments were carried out on 92 dairy farms located in England and Wales.…”

“…Data were collected by seven employed assessors who had received standardised training in the Welfare Quality ® assessment protocols. Inter-assessor reliability testing of the assessors was conducted, and is described in Heath et al (2014). The assessments took place between January and August 2011, each by an individual assessor during a single farm visit, immediately after morning milking, according to the protocol guidelines.…”

“…That Absence of prolonged thirst has been shown in this study, to have such a deterministic role in the overall classification, suggests that the outcomebased, multi-dimensional assessment of welfare that Welfare Quality ® aimed to provide, may be compromised by the system of measure aggregation. While certain 'challenges' have been identified with using the measure aggregation system (Heath et al, 2014), the associated weightings, which derive from both expert opinion and the application of mathematical techniques, have been criticized by de Vries et al (2013a). Therefore, it is the opinion of the authors that rather than evidence of a credible iceberg indicator, the prominent role of Absence of prolonged thirst, in driving the overall classification, represents an unintended consequence of the measure aggregation system, which could potentially be improved with further development.…”

“…Concern over the poor discriminatory ability of the overall Welfare Quality ® classification has recently led Eerdenburg (2013) to propose an adapted version of the measure aggregation system which included, among other changes, the recalibration of the score for Absence of prolonged thirst for what was considered to be an improvement in the overall classification of farms. Missing data were an issue in this study, and are indeed an issue for the Welfare Quality ® measure aggregation system in general (Heath et al, 2014). The methods used here to fill in the missing values were often fairly simple (mean imputation) and could potentially therefore have reduced the variation in some measures between farms.…”

Sequential sampling: a novel method in farm animal welfare assessment

“…First, as it has been shown that there is no strong association between different welfare indicators (Heath et al, 2014) it is likely that it could only be used for a single welfare indicator and may be best suited as part of a targeted lameness surveillance scheme. Second, a challenge posed by welfare outcome-based assessment in general, is the assumption of random sampling.…”