A Conceptual Analytics Model for an Outcome-Driven Quality Management Framework as Part of Professional Healthcare Education

Hervatis,; A, Loe; Barman, Linda; O'Donoghue, JM; Zary, Nabil

doi:10.2196/mededu.4789

Cited by 8 publications

(6 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The application of analytics, defined as the use of data to generate new insights [18], is an emerging approach to study and interpret engagement with mHealth interventions [19]. Van Gemert-Pijnen et al have advanced the application of log data analysis to inform how an intervention works in practice and which components should be improved to yield greater benefit [20-22].…”

Section: Introductionmentioning

confidence: 99%

A Library of Analytic Indicators to Evaluate Effective Engagement with Consumer mHealth Apps for Chronic Conditions: Scoping Review

Pham

Graham

Carrión

et al. 2019

JMIR Mhealth Uhealth

119

129

View full text Add to dashboard Cite

BackgroundThere is mixed evidence to support current ambitions for mobile health (mHealth) apps to improve chronic health and well-being. One proposed explanation for this variable effect is that users do not engage with apps as intended. The application of analytics, defined as the use of data to generate new insights, is an emerging approach to study and interpret engagement with mHealth interventions.ObjectiveThis study aimed to consolidate how analytic indicators of engagement have previously been applied across clinical and technological contexts, to inform how they might be optimally applied in future evaluations.MethodsWe conducted a scoping review to catalog the range of analytic indicators being used in evaluations of consumer mHealth apps for chronic conditions. We categorized studies according to app structure and application of engagement data and calculated descriptive data for each category. Chi-square and Fisher exact tests of independence were applied to calculate differences between coded variables.ResultsA total of 41 studies met our inclusion criteria. The average mHealth evaluation included for review was a two-group pretest-posttest randomized controlled trial of a hybrid-structured app for mental health self-management, had 103 participants, lasted 5 months, did not provide access to health care provider services, measured 3 analytic indicators of engagement, segmented users based on engagement data, applied engagement data for descriptive analyses, and did not report on attrition. Across the reviewed studies, engagement was measured using the following 7 analytic indicators: the number of measures recorded (76%, 31/41), the frequency of interactions logged (73%, 30/41), the number of features accessed (49%, 20/41), the number of log-ins or sessions logged (46%, 19/41), the number of modules or lessons started or completed (29%, 12/41), time spent engaging with the app (27%, 11/41), and the number or content of pages accessed (17%, 7/41). Engagement with unstructured apps was mostly measured by the number of features accessed (8/10, P=.04), and engagement with hybrid apps was mostly measured by the number of measures recorded (21/24, P=.03). A total of 24 studies presented, described, or summarized the data generated from applying analytic indicators to measure engagement. The remaining 17 studies used or planned to use these data to infer a relationship between engagement patterns and intended outcomes.ConclusionsAlthough researchers measured on average 3 indicators in a single study, the majority reported findings descriptively and did not further investigate how engagement with an app contributed to its impact on health and well-being. Researchers are gaining nuanced insights into engagement but are not yet characterizing effective engagement for improved outcomes. Raising the standard of mHealth app efficacy through measuring analytic indicators of engagement may enable greater confidence in the causal impact of apps on improved chronic health and well-being.

show abstract

Section: Introductionmentioning

confidence: 99%

A Library of Analytic Indicators to Evaluate Effective Engagement with Consumer mHealth Apps for Chronic Conditions: Scoping Review

Pham

Graham

Carrión

et al. 2019

JMIR Mhealth Uhealth

119

129

View full text Add to dashboard Cite

show abstract

“…If we had been able to analyze data extracted from log-in dates, visited pages, and time passed on each page per connection, our interpretation would have been more meaningful. To improve a Web-based training such as this, more information is needed about how the residents use the tutorial, and data collection needs to be adapted to reflect modifications in the program when they take place [41]. With the increase in Web-based interventions, the potential for data extraction is growing exponentially [42].…”

Section: Discussionmentioning

confidence: 99%

Teaching Shared Decision Making to Family Medicine Residents: A Descriptive Study of a Web-Based Tutorial

et al. 2016

View full text Add to dashboard Cite

BackgroundDECISION+2, a Web-based tutorial, was designed to train family physicians in shared decision making (SDM) regarding the use of antibiotics for acute respiratory infections (ARIs). It is currently mandatory for second-year family medicine residents at Université Laval, Quebec, Canada. However, little is known about how such tutorials are used, their effect on knowledge scores, or how best to assess resident participation.ObjectiveThe objective of our study was to describe the usage of this Web-based training platform by family medicine residents over time, evaluate its effect on their knowledge scores, and identify what kinds of data are needed for a more comprehensive analysis of usage and knowledge acquisition.MethodsWe identified, collected, and analyzed all available data about participation in and current usage of the tutorial and its before-and-after 10-item knowledge test. Residents were separated into 3 log-in periods (2012-2013, 2013-2014, and 2014-2015) depending on the day of their first connection. We compared residents’ participation rates between entry periods (Cochran-Armitage test), assessed the mean rank of the difference in total scores and category scores between pre- and posttest (Wilcoxon signed-rank test), and compared frequencies of each. Subsequent to analyses, we identified types of data that would have provided a more complete picture of the usage of the program and its effect on knowledge scores.ResultsThe tutorial addresses 3 knowledge categories: diagnosing ARIs, treating ARIs, and SDM regarding the use of antibiotics for treating ARIs. From July 2012 to July 2015, all 387 second-year family medicine residents were eligible to take the Web-based tutorial. Out of the 387 eligible residents, 247 (63.8%) logged in at least once. Their participation rates varied between entry periods, most significantly between the 2012-2013 and 2013-2014 cohorts (P=.006). For the 109 out of 387 (28.2%) residents who completed the tutorial and both tests, total and category scores significantly improved between pre- and posttest (all P values <.001). However, the frequencies of those answering correctly on 2 of the 3 SDM questions did not increase significantly (P>.99, P=.25). Distribution of pre- or posttest total and category scores did not increase between entry periods (all P values >.1). Available data were inadequate for evaluating the associations between the tutorial and its impact on the residents’ scores and therefore could tell us little about its effect on increasing their knowledge.ConclusionResidents’ use of this Web-based tutorial appeared to increase between entry periods following the changes to the SDM program, and the tutorial seemed less effective for increasing SDM knowledge scores than for diagnosis or treatment scores. However, our results also highlight the need to improve data availability before participation in Web-based SDM tutorials can be properly evaluated or knowledge scores improved.

show abstract

“…Education Feedback The objective of the study (Hervatis, V., et. al., 2015) varies among various research communities.…”

Section: Implication Of Big Data In the Context Of The Education Syst...mentioning

confidence: 99%

Working and Impact of Parliamentary Committees in the UK and Bangladesh: A Theoretical Analysis

Chowdhury

2022

Dhaka Uni. Law. J.

View full text Add to dashboard Cite

show abstract

A Conceptual Analytics Model for an Outcome-Driven Quality Management Framework as Part of Professional Healthcare Education

Cited by 8 publications

References 13 publications

A Library of Analytic Indicators to Evaluate Effective Engagement with Consumer mHealth Apps for Chronic Conditions: Scoping Review

A Library of Analytic Indicators to Evaluate Effective Engagement with Consumer mHealth Apps for Chronic Conditions: Scoping Review

Teaching Shared Decision Making to Family Medicine Residents: A Descriptive Study of a Web-Based Tutorial

Working and Impact of Parliamentary Committees in the UK and Bangladesh: A Theoretical Analysis

Contact Info

Product

Resources

About