Using electronic health record audit log data for research: insights from early efforts

Abstract

Electronic health record audit logs capture a time-sequenced record of clinician activities while using the system. Audit log data therefore facilitate unobtrusive measurement at scale of clinical work activities and workflow as well as derivative, behavioral proxies (eg, teamwork). Given its considerable research potential, studies leveraging these data have burgeoned. As the field has matured, the challenges of using the data to answer significant research questions have come into focus. In this Perspective, we draw on our research experiences and insights from the broader audit log literature to advance audit log research. Specifically, we make 2 complementary recommendations that would facilitate substantial progress toward audit log-based measures that are: (1) transparent and validated, (2) standardized to allow for multisite studies, (3) sensitive to meaningful variability, (4) broader in scope to capture key aspects of clinical work including teamwork and coordination, and (5) linked to patient and clinical outcomes.

Using event logs to observe interactions with electronic health records: an updated scoping review shows increasing use of vendor-derived measures

“…More work is also needed to ensure measure validity. 126 Studies should clearly demonstrate the criterion validity of measures they employ (i.e., that values derived from event logs match those from gold-standard methods such as direct observation), as well as content validity (i.e., that measures include all relevant EHR activity) and construct validity (i.e., that measures relate to a construct/theory/trait of interest such as documentation burden). The number of unique definitions of EHR time outside scheduled hours ( Table 1 ) demonstrates the difficulty of operationalizing the construct of work outside work.…”

Identifying and Addressing Barriers to Implementing Core Electronic Health Record Use Metrics for Ambulatory Care: Virtual Consensus Conference Proceedings

“…We identified common themes across groups including an urgent need for standardized definitions of EHR use measure elements, which aligns with the existing literature. 5 6 For example, priority was raised for defining what a clinical day/shift or clinical role included, operationalizing metrics to evaluate EHR active time, determining the scope and complexity of inbox management tasks, and establishing units of time latency, task, and attention including an ontology of terms to standardize future work. Understanding the interaction between team members, and the interdisciplinary aspects of care, which are not fully captured in current metrics, was also offered as an opportunity for action when designing and implementing EHR metrics.…”

“… 2 Although current vendor-derived metrics seek to offer actionable benchmarks on EHR use, they face validity and reliability concerns due to limited transparency, availability, accessibility, and standardization. 6 For example, agreement on how to conceptualize time spent in the EHR outside of time a clinician is scheduled—one construct that has been linked to clinician burnout 7 —has encountered limitations including generalizability across vendors and may not effectively separate time spent on direct patient care from time strictly dedicated to the EHR. 8 Scientifically sound evaluation will require stakeholder consensus on the optimal approaches to harness these collective resources to ensure that metrics are meaningful and useful to all stakeholders.…”

“…2 Although current vendor-derived metrics seek to offer actionable benchmarks on EHR use, they face validity and reliability concerns due to limited transparency, availability, accessibility, and standardization. 6 For example, agreement on how to conceptualize time spent in the EHR outside of time a clinician is scheduled -one construct that 4 This article is protected by copyright. All rights reserved.…”

Experiences, barriers and expectations regarding current patient monitoring systems among ICU nurses in a University Hospital in Lebanon: a qualitative study

“…There are significant limitations to be addressed, nevertheless, such as alert fatigue brought on by a high number of false alarms, difficult user interfaces, difficulties with interoperability, and a lack of full integration with electronic health record systems ( 8 ). Additionally, according to Kannampallil and Adler-Milstein ( 9 ), ICU nurses anticipate better system use, streamlined workflows, individualized alarm settings, seamless data sharing, and greater interprofessional communication.…”