A qualitative study exploring the acceptability of the McNulty-Zelen design for randomised controlled trials evaluating educational interventions

Abstract

BackgroundTraditional randomised controlled trials evaluating the effect of educational interventions in general practice may produce biased results as participants know they are being evaluated. We aimed to explore the acceptability of a McNulty-Zelen Cluster Randomised Control Trial (CRT) design which conceals from educational participants that they are in a RCT. Consent is obtained from a trusted third party considered appropriate to give consent on participants’ behalf, intervention practice staff then choose whether to attend the offered education as would occur with normal continuing professional development.MethodsWe undertook semi structured telephone interviews in England with 16 general practice (GP) staff involved in a RCT evaluating an educational intervention aimed at increasing chlamydia screening tests in general practice using the McNulty-Zelen design, 4 Primary Care (PC) Research Network officers, 5 Primary Care Trust leads in Public or sexual health, and one Research Ethics committee Chair. Interviews were undertaken by members of the original intervention evaluation McNulty-Zelen design RCT study team. These experienced qualitative interviewers used an agreed semi-structured interview schedule and were careful not to lead the participants. To further mitigate against bias, the data analysis was undertaken by a researcher (CR) not involved in the original RCT.ResultsWe reached data saturation and found five main themes;Support for the design: All found the McNulty-Zelen design acceptable because they considered that it generated more reliable evidence of the value of new educational interventions in real life GP settings.Lack of familiarity with study design: The design was novel to all. GP staff likened the evaluation using the McNulty–Zelen design to audit of their activities with feedback, which were to them a daily experience and therefore acceptable.Ethical considerations: Research stakeholders considered the consent procedure should be very clear and that these trial designs should go through at least a proportionate ethical review. GP staff were happy for the PCT leads to give consent on their behalf.GP research capacity and trial participation: GP staff considered the design increased generalisability, as staff who would not normally volunteer to participate in research due to perceived time constraints and paperwork might do so.Design ‘worth it’: All interviewees agreed that the advantages of the “more accurate” or “truer” results and information gained about uptake of workshops within Primary Care Trusts (PCTs) outweighed any disadvantages of the consent procedure.DiscussionOur RCT was evaluating the effect of an educational intervention to increase chlamydia screening tests in general practices where there was routine monitoring of testing rates; our participants may have been less enthusiastic about the design if it had been evaluating a more controversial educational area, or if data monitoring was not routine.ImplicationsThe McNulty-Zelen design should be considered for the evaluat...

Saturation in qualitative research: exploring its conceptualization and operationalization

“…Quite frequently, studies that adopt the criterion of saturation propose at the same time a prior sample size (e.g. McNulty et al 2015 ; Long-Sutehall et al 2011 ). In a similar way, Niccolai et al ( 2016 ) sought saturation during their analysis, but also state (p. 843) that: An a priori sample size of 30 to 40 was selected based on recommendations for qualitative studies of this nature… and the anticipated complexity and desired level of depth for our research questions.…”

Theoretical Saturation in Grounded Theory Studies: An Evaluative Tool

“…A pre-determined number of interviews is not new in qualitative research. Many studies show saturation as the core of their research and analysis process yet also shared prescribed number of a sample size to attain saturation (McNulty et al, 2015;Long-Sutehall et al, 2011;Niccolai et al, 2016;Fusch and Ness, 2015). Marshall et al (2013), in their systematic review of qualitative literature, revealed that experts suggested different numbers of participants for qualitative studies.…”

Effectiveness of behavioural interventions to reduce urinary tract infections and Escherichia coli bacteraemia for older adults across all care settings: a systematic review

“…Whereas it is not always possible to conduct full RCTs with behavioural interventions, the use of control groups and appropriate randomization procedures should be considered where possible, as well as sufficiently powered samples. Future studies may want to consider using the McNultyeZelen design which allows for randomization and blinded conditions by seeking proxy consent or a steppedwedge design [55,56]. This allows for a strong methodological evaluation of an intervention used in routine healthcare.…”