Next Steps in the Implementation of Learning Analytics in Medical Education: Consensus From an International Cohort of Medical Educators

Thoma, Brent; Warm, Eric J.; Hamstra, Stanley J.; Cavalcanti, Rodrigo B.; Pusic, Martin; Shaw, Tim; Verma, Amol A.; Frank, Jason R.; Hauer, Karen E.

doi:10.4300/jgme-d-19-00493.1

Cited by 25 publications

(30 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…22 From an educational perspective, LHSs, and the EHRs that inform them, may provide an opportunity to enhance clinical training. 24,25 Medical students could both contribute to and learn from the new knowledge derived from information in EHRs. Future research could then examine the utility of using EHRs for educational purposes early in medical school and their subsequent impact on future practice behaviors.…”

Section: Discussionmentioning

confidence: 99%

Medical Student Experiences With Electronic Health Records Nationally: A Longitudinal Analysis Including School-Level Effects

et al. 2021

View full text Add to dashboard Cite

Purpose Increasingly, medical school graduates have been expected to be competent in the knowledge and skills associated with effective electronic health record (EHR) use. Yet little is known about how student experiences with EHRs have changed over time or how these trends vary by medical school. This study examined shifts in U.S. medical student interactions with EHRs during their clinical education, 2012–2016, and how these interactions varied by clerkship within and across medical schools. Method Data came from an online survey about EHR use administered to medical students after completing the Step 2 Clinical Knowledge portion of the United States Medical Licensing Examination. For a sample of 17,202 medical students from 140 U.S.-based Liaison Committee on Medical Education accredited medical school campuses, multilevel modeling techniques were used to estimate overall and school-specific trends in student access to and entry of information into EHRs for 7 core inpatient clerkships. Results Results showed upward trajectories in likelihood of student EHR experiences, with smaller increases found for information entry compared with access. These trends varied by inpatient clerkship rotation, with some disciplines exhibiting more rapid increases than others. For both access and information entry, estimated clerkship-specific trajectories differed by medical school in terms of size and direction. For all clerkships, greater school-to-school variation in the likelihood of student entry, compared with student access, remained at the end of the study period. Conclusions Increases in medical student interactions with EHRs suggest a growing commitment to educating students on safe and effective EHR use. Nonetheless, at some schools and in some clerkships, students may receive inadequate educational opportunities to practice using EHRs. In turn, medical students may be differentially prepared to effectively engage with EHRs upon entering residency, particularly with the knowledge and skills needed to effectively document and transmit information in EHRs.

show abstract

Section: Discussionmentioning

confidence: 99%

Medical Student Experiences With Electronic Health Records Nationally: A Longitudinal Analysis Including School-Level Effects

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Licensing and regulatory bodies should look for similar incentives and disincentives in the structures and policies they oversee. In the OPA system, educators collect thousands of data points over time from multiple sources, plot these on a dashboard using learning analytics, 39 and pair these plots with narrative assessment. 36,40,41 A CCC then reviews these data to create learning plans for residents and their coaches to consider, and to make summary judgements about advancement, promotion, and graduation.…”

Section: Creating a True Medical Education Continuummentioning

confidence: 99%

“…At each stage of education and practice the student/physician would grant access to all entities that could provide data on performance. Using learning analytics, 39,43,44 database managers would connect and display information over time that could be used by the student/physician and their assigned or chosen coaches and review committees. Each level of training and practice would consist of the data relevant for that level (e.g., tests for medical students, case logs for residents, quality outcomes and patient satisfaction for attending physicians), with each element mapped to an organizing principle such as CanMeds roles, 4 EPAs 6,45-47 or milestones 48 for longitudinal performance tracking over time.…”

Section: Traveling Across the Continuum: Education Passportsmentioning

confidence: 99%

The education passport: connecting programmatic assessment across learning and practice

et al. 2022

View full text Add to dashboard Cite

Competency-based medical education (CBME) shifts us from static assessment of learning to developmental assessment for learning. However, implementation challenges associated with CBME remain a major hurdle, especially after training and into practice. The full benefit of developmental assessment for learning over time requires collaboration, cooperation, and trust among learners, regulators, and the public that transcends each individual phase. The authors introduce the concept of an “Education Passport” that provides evidence of readiness to travel across the boundaries between undergraduate medical education, graduate medical education, and the expanse of practice. The Education Passport uses programmatic assessment, a process of collecting numerous low stakes assessments from multiple sources over time, judging these data using criterion-referencing, and enhancing this with coaching and competency committees to understand, process, and accelerate growth without end. Information in the Passport is housed on a cloud-based server controlled by the student/physician over the course of training and practice. These data are mapped to various educational frameworks such Entrustable Professional Activities or milestones for ease of longitudinal performance tracking. At each stage of education and practice the student/physician grants Passport access to all entities that can provide data on performance. Database managers use learning analytics to connect and display information over time that are then used by the student/physician, their assigned or chosen coaches, and review committees to maintain or improve performance. Global information is also collected and analyzed to improve the entire system of learning and care. Developing a true continuum that embraces performance and growth will be a long-term adaptive challenge across many organizations and jurisdictions and will require coordination from regulatory and national agencies. An Education Passport could also serve as an organizing tool and will require research and high-value communication strategies to maximize public trust in the work.

show abstract

“…Whether or not an analytics approach is used, the security of trainee assessment data is a concern. 43 While this problem has always existed, modern systems may be more prone to hacking and other digital data security breaches because they are stored in larger databases with numerous distributed access points. Additionally, the assessment data that we collect are increasingly detailed and sensitive.…”

Section: Data Securitymentioning

confidence: 99%

From Utopia Through Dystopia: Charting a Course for Learning Analytics in Competency-Based Medical Education

2021

Self Cite

View full text Add to dashboard Cite

The transition to the assessment of entrustable professional activities as part of competency-based medical education (CBME) has substantially increased the number of assessments completed on each trainee. Many CBME programs are having difficulty synthesizing the increased amount of assessment data. Learning analytics are a way of addressing this by systematically drawing inferences from large datasets to support trainee learning, faculty development, and program evaluation. Early work in this field has tended to emphasize the significant potential of analytics in medical education. However, concerns have been raised regarding data security, data ownership, validity, and other issues that could transform these dreams into nightmares. In this paper, the authors explore these contrasting perspectives by alternately describing utopian and dystopian futures for learning analytics within CBME. Seeing learning analytics as an important way to maximize the value of CBME assessment data for organizational development, they argue that their implementation should continue within the guidance of an ethical framework.

show abstract

Next Steps in the Implementation of Learning Analytics in Medical Education: Consensus From an International Cohort of Medical Educators

Cited by 25 publications

References 27 publications

Medical Student Experiences With Electronic Health Records Nationally: A Longitudinal Analysis Including School-Level Effects

Medical Student Experiences With Electronic Health Records Nationally: A Longitudinal Analysis Including School-Level Effects

The education passport: connecting programmatic assessment across learning and practice

From Utopia Through Dystopia: Charting a Course for Learning Analytics in Competency-Based Medical Education

Contact Info

Product

Resources

About