Study design and protocol for a comprehensive evaluation of a UK massive open online course (MOOC) on quality improvement in healthcare

Abstract

IntroductionMassive open online courses (MOOCs) offer a flexible approach to online and distance learning, and are growing in popularity. Several MOOCs are now available, to help learners build on their knowledge in a number of healthcare topics. More research is needed to determine the effectiveness of MOOCs as an online education tool, and explore their long-term impact on learners’ professional practice. We present a protocol describing the design of comprehensive, mixed-methods evaluation of a MOOC, ‘QualityImprovement (QI) inHealthcare: the Case for Change’, which aims to improve learner’s knowledge and understanding of QI approaches in healthcare, and to increase their confidence in participating, and possibly leading QI projects.Methods and analysisA pre-post study design using quantitative and qualitative methods will be used to evaluate the QI MOOC. Different elements of the RE-AIM (reach, effectiveness and maintenance) and Kirkpatrick (reaction, learning and behaviour) models will be used to guide the evaluation. All learners who register for the course will be invited to participate in the QI MOOC evaluation study. Those who consent will be asked to complete a presurvey to assess baseline QI knowledge (self-report and objective) and perceived confidence in engaging in QI activities. On completion of the course, participants will complete a postsurvey measuring again knowledge and perceived confidence. Feedback on the course content and how it can be improved. A subset of participants will be invited to take part in a follow-up qualitative interview, 3 months after taking the course, to explore in depth how the MOOC impacted their behaviour in practice.Ethics and disseminationThe study has been approved by the University of Bath Human Research Ethics Committee (reference: 2958). Study findings will be published in peer-reviewed journals, and disseminated at conference and departmental presentations, and more widely using social media, microblogging sites and periodicals aimed at healthcare professionals.

Massive open online courses for nurses’ and healthcare professionals’ continuous education: a scoping review

“…With regard to the certification of the learning outcomes achieved, the application of several outcome evaluation methods, such as quantitative (Canavese et al 2020) and qualitative (Smith‐Lickess et al 2019) and through browser and social media (Harvey et al 2017), seems to be consistent with the andragogy theory (Berland et al 2020), where the needs, aims and learning styles of adults’ learners should be considered. However, the variability of outcomes measured may be a potential limitation in investigating the effectiveness of MOOCs when performing, for example, systematic reviews or meta‐analyses.…”

“…Universities were reported as sole providers in 11 studies (Beer 2019; Fricton et al 2015; Gitlin & Hodgson 2016; Goldberg & Crocombe 2017; Hoedebecke et al 2018; Jia et al 2019; Liyanagunawardena & Williams 2014; Milligan & Littlejohn 2016; Pickering & Swinnerton 2017; Sitzman et al 2016; Subhi et al 2014) and in association with other institutions in 10 studies (Bendezu‐Quispe et al 2017; Canavese et al 2020; Evans et al 2017; King et al 2014; Medina‐Presentado et al 2017; Scott et al 2019; Smith‐Lickess et al 2019; Sneddon et al 2018; Warugaba et al 2016; Zubala et al 2019).…”

“…According to the study findings, MOOCs are mainly offered using a platform such as Coursera (Evans et al 2017; Fricton et al 2015; Gitlin & Hodgson 2016; Subhi et al 2014; Warugaba et al 2016; Wewer Albrechtsen et al 2017) and FutureLearn (Beer 2019; Charani et al 2019; Pickering & Swinnerton 2017; Smith‐Lickess et al 2019; Sneddon et al 2018), reported individually or in combination (Bendezu‐Quispe et al 2017; Liyanagunawardena & Williams 2014).…”

“…questionnaire, quiz; Canavese et al 2020; Charani et al 2019), while a qualitative assessment (e.g. essay) was reported in eight studies (Beer 2019; Chan et al 2015; Gitlin & Hodgson 2016; Goldberg & Crocombe 2017; Sitzman et al 2016; Smith‐Lickess et al 2019; Warugaba et al 2016; Zubala et al 2019). Moreover, four studies evaluated the degree of engagement by collecting data from browsers and social media (Chan et al 2015; Harvey et al 2014; Harvey et al 2017; Sitzman et al 2016).…”

“…Moreover, four studies evaluated the degree of engagement by collecting data from browsers and social media (Chan et al 2015; Harvey et al 2014; Harvey et al 2017; Sitzman et al 2016). The final certificate for MOOC attendance has been documented to be achieved on a voluntary basis in 10 studies, although this was without obtaining credits (Charani et al 2019; Fricton et al 2015; Gitlin & Hodgson 2016; Harvey et al 2014; Harvey et al 2017; Magaña‐Valladares et al 2018a; Pérez‐Moreno et al 2018; Smith‐Lickess et al 2019; Sneddon et al 2018; Stark & Pope 2014). For the remainder, when reported, different forms of evaluation were provided mainly with questionnaires, assessing not only the knowledge gained (Canavese et al 2020) but also the degree of engagement experienced (Chan et al 2015).…”