Using event logs to observe interactions with electronic health records: an updated scoping review shows increasing use of vendor-derived measures

Abstract

Objective The aim of this article is to compare the aims, measures, methods, limitations, and scope of studies that employ vendor-derived and investigator-derived measures of electronic health record (EHR) use, and to assess measure consistency across studies. Materials and Methods We searched PubMed for articles published between July 2019 and December 2021 that employed measures of EHR use derived from EHR event logs. We coded the aims, measures, methods, limitations, and scope of each article and compared articles employing vendor-derived and investigator-derived measures. Results One hundred and two articles met inclusion criteria; 40 employed vendor-derived measures, 61 employed investigator-derived measures, and 1 employed both. Studies employing vendor-derived measures were more likely than those employing investigator-derived measures to observe EHR use only in ambulatory settings (83% vs 48%, P = .002) and only by physicians or advanced practice providers (100% vs 54% of studies, P < .001). Studies employing vendor-derived measures were also more likely to measure durations of EHR use (P < .001 for 6 different activities), but definitions of measures such as time outside scheduled hours varied widely. Eight articles reported measure validation. The reported limitations of vendor-derived measures included measure transparency and availability for certain clinical settings and roles. Discussion Vendor-derived measures are increasingly used to study EHR use, but only by certain clinical roles. Although poorly validated and variously defined, both vendor- and investigator-derived measures of EHR time are widely reported. Conclusion The number of studies using event logs to observe EHR use continues to grow, but with inconsistent measure definitions and significant differences between studies that employ vendor-derived and investigator-derived measures.

Anesthesia Clinical Workload Estimated From Electronic Health Record Documentation vs Billed Relative Value Units

“…Audit logs capture clinician click actions within the EHR that result in the display or modification of patient data and are mandated for security purposes by the Health Insurance Portability and Accountability Act. Each click action generates a record of what action was performed, time stamp of the action, patient on which the action occurred, and user responsible for the action (eFigure 1 in Supplement 1); because most clinical work is documented in the EHR, audit logs have previously been used to measure clinical workflow and workload …”

“…Nearly as much EHR use occurred in the preoperative period (Figure 1A) as the intraoperative period. For each encounter, clinicians spent a median (IQR) of 11 (6)(7)(8)(9)(10)(11)(12)(13)(14)(15)(16)(17)(18)(19) minutes using the EHR and performing a median (IQR) of 66 (45-92) EHR-based actions in the preoperative period. Preoperative actions typically involved review of patient data.…”

The role of organizations in shaping physician use of electronic health records

“…Primary care specialties include internal medicine, family medicine, pediatrics, obstetrics and gynecology, geriatrics, and sports medicine. Data on EHR use behaviors for these physicians were extracted at the physician‐week level of observation from the Epic Signal data warehouse 5,10,18,23 . Our study period covered September 2020 through May 2021.…”

“…Guiding enhanced organizational efforts around optimizing the EHR use experience requires new definition and measurement 4 . Most literature on EHR burden defines “better” as lower with respect to time spent in the EHR 18 . It could also be operationalized as the use of vendor efficiency tools, for example, templates, macros, and copy/paste, designed to support documentation completion.…”

Identifying and Addressing Barriers to Implementing Core Electronic Health Record Use Metrics for Ambulatory Care: Virtual Consensus Conference Proceedings

“…1 Since then, there have been more than 100 peer-reviewed publications addressing EHR burden, garnering increased attention from across stakeholder groups. 2,3 Intended to quantify practice efficiency, teamwork and other contributors to professional well-being, and promote cross-study comparisons, 1 wide-scale implementation of core metrics in the research setting as well as routine operations assessment have faced challenges across practice settings. 4 Precise, reliable, valid metrics that are cost-effective, and require reasonable implementation time and effort are needed to drive EHR improvements.…”