Measuring Collaboration Through Concurrent Electronic Health Record Usage: Network Analysis Study

Abstract

Background Collaboration is vital within health care institutions, and it allows for the effective use of collective health care worker (HCW) expertise. Human-computer interactions involving electronic health records (EHRs) have become pervasive and act as an avenue for quantifying these collaborations using statistical and network analysis methods. Objective We aimed to measure HCW collaboration and its characteristics by analyzing concurrent EHR usage. Methods By extracting concurrent EHR usage events from audit log data, we defined concurrent sessions. For each HCW, we established a metric called concurrent intensity, which was the proportion of EHR activities in concurrent sessions over all EHR activities. Statistical models were used to test the differences in the concurrent intensity between HCWs. For each patient visit, starting from admission to discharge, we measured concurrent EHR usage across all HCWs, which we called temporal patterns. Again, we applied statistical models to test the differences in temporal patterns of the admission, discharge, and intermediate days of hospital stay between weekdays and weekends. Network analysis was leveraged to measure collaborative relationships among HCWs. We surveyed experts to determine if they could distinguish collaborative relationships between high and low likelihood categories derived from concurrent EHR usage. Clustering was used to aggregate concurrent activities to describe concurrent sessions. We gathered 4 months of EHR audit log data from a large academic medical center’s neonatal intensive care unit (NICU) to validate the effectiveness of our framework. Results There was a significant difference (P<.001) in the concurrent intensity (proportion of concurrent activities: ranging from mean 0.07, 95% CI 0.06-0.08, to mean 0.36, 95% CI 0.18-0.54; proportion of time spent on concurrent activities: ranging from mean 0.32, 95% CI 0.20-0.44, to mean 0.76, 95% CI 0.51-1.00) between the top 13 HCW specialties who had the largest amount of time spent in EHRs. Temporal patterns between weekday and weekend periods were significantly different on admission (number of concurrent intervals per hour: 11.60 vs 0.54; P<.001) and discharge days (4.72 vs 1.54; P<.001), but not during intermediate days of hospital stay. Neonatal nurses, fellows, frontline providers, neonatologists, consultants, respiratory therapists, and ancillary and support staff had collaborative relationships. NICU professionals could distinguish high likelihood collaborative relationships from low ones at significant rates (3.54, 95% CI 3.31-4.37 vs 2.64, 95% CI 2.46-3.29; P<.001). We identified 50 clusters of concurrent activities. Over 87% of concurrent sessions could be described by a single cluster, with the remaining 13% of sessions comprising multiple clusters. Conclusions Leveraging concurrent EHR usage workflow through audit logs to analyze HCW collaboration may improve our understanding of collaborative patient care. HCW collaboration using EHRs could potentially influence the quality of patient care, discharge timeliness, and clinician workload, stress, or burnout.

Using event logs to observe interactions with electronic health records: an updated scoping review shows increasing use of vendor-derived measures

“… 75 , 93 , 94 , 113 Only one of the experimental studies was a randomized controlled trial. 75 Across both observational and experimental studies, 41 articles compared EHR use across different groups of users including comparisons by specialty (14 studies), 4 , 7 , 25 , 29 , 31 , 36 , 42 , 48 , 51 , 58 , 59 , 85 , 105 , 115 clinical role (12), 24 , 26 , 33 , 37 , 46 , 65 , 76 , 95 , 99 , 106 , 108 , 115 gender (8), 19 , 34 , 47 , 49 , 66 , 68 , 70 , 106 year in residency (8), 31 , 32 , 35 , 39 , 52 , 56 , 57 , 95 organization (3), 24 , 44 , 75 and country (1). 27 Vendor-measure studies were more likely than investigator-measure studies to make such comparisons of EHR use by user group (65% vs 25...…”

“…Eight articles reported the results of measure validation, with no difference in reporting between vendor-measure and investigator-measure studies (10% vs 5%, P = .473). 6 , 53 , 79 , 80 , 83 , 85 , 102 , 103 These articles compared resident duty hours derived from EHR logs with self-reported or Global Positioning System tracked hours (3 articles), 80 , 83 , 102 patient-clinician relationships derived from logs with those from COVID-19 contact tracers or self-report (2 articles), 85 , 103 EHR time derived from logs with self-report (2 articles), 6 , 53 and EHR login time derived from logs with those observed through screen recording (1 article). 79 Five additional articles referenced previous validations for measures of EHR time including articles referencing Epic’s, 28 Cerner’s, 29 Arndt et al’s, 66 , 124 and “other EHR vendors” methods.…”

“…Reviewed articles mentioned several limitations of EHR log research ( Supplementary eTable 4 ). Three of the most frequently mentioned limitations echo those observed in the prior review of audit log research 11 : EHR logs do not provide a full view of clinical activity which can involve physical and digital interactions outside the EHR (22 articles), 3 , 26 , 34 , 43 , 55 , 57 , 58 , 61 , 65 , 68 , 73 , 74 , 84–86 , 89 , 91 , 92 , 95 , 103 , 109 , 112 qualitative methods are needed to better understand the context and motivation for observed work (15 articles), 32 , 46 , 53 , 62 , 64 , 68 , 72 , 80 , 85 , 91 , 96 , 97 , 101 , 115 , 118 and logs may not contain enough detail to observe complex workflows (13 articles). 4 , 26 , 32 , 43 , 51 , 56 , 59 , 69 , 75 , 81 , 89 , 90 ,…”

Learning from Non-Routine Events and Teamwork in Intensive Care Units: Challenges and Opportunities

“…EHR access logs capturing the human-computer interactions in EHRs are one of the big data sources for investigating EHR tasks and corresponding utilization workflows to complete the tasks. Our recent publication demonstrates the effectiveness of using EHR access logs in learning complex and time-dependent tasks performed by healthcare professionals [9,10]. Building connections between NREs and EHRs would enable the investigation of NRE contexts, for instance, how EHRs support the completion of timedependent tasks.…”