Applying Qualitative Comparative Analysis (QCA) to evaluate a public health policy initiative in the North East of England

Abstract

This paper presents a Qualitative Comparative Analysis (QCA) analysis of data produced as part of the evaluation a Nation Health Service commissioned intervention in the North East of England. QCA is a case-oriented method that allows systematic comparison of cases as configurations of set memberships based on their attributes and the relationship of these to particular outcomes. QCA provides an alternative to conventional quantitative approaches which are generally concerned with isolating the independent effect of one variable whilst controlling the influence of others. Instead, QCA allows for interactions between multiple attributes and recognises that the same outcomes may be generated by different configurations of attributes. The intervention evaluated provided case management for individuals who were out of work due to ill health, and had been for three years or more. It aimed to improve the health of individuals and move them closer to the labour market. The intervention and a comparison group were assessed at base line (T1), after 3 months — (T2) after 6 months (end of the intervention — T3) and after 9 months (three months post intervention — T4). The size of the respective populations at each time point were, Intervention group at T1, N = 131, T2, N = 44, T3, N = 79, T4, N = 95. Comparison group at T1, N = 229, T2, N = 188, T3, N = 166, T4, N = 154. General health was measured using EQ5-D (a standardised instrument for use as a measure of general health outcome) and SF-8. Two condition specific measures were included: the Hospital Anxiety and Depression Scale (HADS) and the Nordic Musculoskeletal questionnaire. Data was also collected on socio demographics (gender, age, housing tenure), social capital (contact with family and friends and participation with the wider community), and work history (previous jobs, time spent in the job, time spent on sickness absence). The aim of the QCA analysis was to identify whether individuals with certain characteristics or combinations of characteristics benefited from the intervention. In order to do this the cases were sorted according to whether their EQ-5D VAS (Visual Analogue Scale) scores (a self rated measure of general health) narrowed or did not narrow towards the population norm for the measure between baseline (T1) and (T4) 9 months (three months post intervention — T4). Cases which narrowed towards the UK population norm of 82.48 were judged to be experiencing a health improvement whilst those whose scores did not narrow towards the norm were judged to not have experienced a health improvement. A crisp set (cs) QCA analysis was then performed. The paper assesses the benefits of using QCA, and asks whether it can provide a viable and practical tool for social policy evaluations.

Natural Experiments: An Overview of Methods, Approaches, and Contributions to Public Health Intervention Research

“…Qualitative studies conducted alongside the quantitative evaluation of Scotland’s smoke-free legislation have been used to assess compliance with the intervention (24) and to identify a range of secondary outcomes such as changes in smoking behavior within the home (60). Qualitative methods for identifying the effects of interventions have also been proposed, but further studies are needed to establish their validity and usefulness (63, 72, 77).…”

Symmetric and asymmetric motivations for compliance and violation: A crisp set qualitative comparative analysis ofChinese farmers

“…Regression analysis may be applied to a similar end; Vis () and Warren et al . () contrast QCA with regression analysis, highlighting the strengths and weaknesses of both approaches.…”

Voluntary programmes for building retrofits: opportunities, performance and challenges

“…In a recent article, Barbara Vis (2012) has compared fsQCA with Regression Analysis by applying the two data analysis techniques to a single dataset. She finds that fsQCA is better capable to understand and make visible complex relations than regression analysis (for a comparable argument, see Warren, Wistow, & Bambra, 2013).…”