Attributing Patients to Pediatric Residents Using Electronic Health Record Features Augmented with Audit Logs

Abstract

Objective Patient attribution, or the process of attributing patient-level metrics to specific providers, attempts to capture real-life provider–patient interactions (PPI). Attribution holds wide-ranging importance, particularly for outcomes in graduate medical education, but remains a challenge. We developed and validated an algorithm using EHR data to identify pediatric resident PPIs (rPPIs). Methods We prospectively surveyed residents in three care settings to collect self-reported rPPIs. Participants were surveyed at the end of primary care clinic, emergency department (ED), and inpatient shifts, shown a patient census list, asked to mark the patients with whom they interacted, and encouraged to provide a short rationale behind the marked interaction. We extracted routine EHR data elements, including audit logs, note contribution, order placement, care team assignment, and chart closure, and applied a logistic regression classifier to the data to predict rPPIs in each care setting. We also performed a comment analysis of the resident-reported rationales in the inpatient care setting to explore perceived patient interactions in a complicated workflow. Results We surveyed 81 residents over 111 shifts and identified 579 patient interactions. Among EHR extracted data, time-in-chart was the best predictor in all three care settings (primary care clinic: odds ratio [OR] = 19.36, 95% confidence interval [CI]: 4.19–278.56; ED: OR = 19.06, 95% CI: 9.53–41.65' inpatient: OR = 2.95, 95% CI: 2.23–3.97). Primary care clinic and ED specific models had c-statistic values > 0.98, while the inpatient-specific model had greater variability (c-statistic = 0.89). Of 366 inpatient rPPIs, residents provided rationales for 90.1%, which were focused on direct involvement in a patient's admission or transfer, or care as the front-line ordering clinician (55.6%). Conclusion Classification models based on routinely collected EHR data predict resident-defined rPPIs across care settings. While specific to pediatric residents in this study, the approach may be generalizable to other provider populations and scenarios in which accurate patient attribution is desirable.

Using event logs to observe interactions with electronic health records: an updated scoping review shows increasing use of vendor-derived measures

“… 11 Aims were not mutually exclusive so each study could have more than 1 aim. While all vendor-measure studies measured an amount of EHR use (e.g., volume or duration), 2–4 , 6 , 19 , 24–58 a third (33%) of investigator-measure studies did not, 60 , 68 , 74 , 78 , 80 , 82–87 , 89 , 91 , 92 , 98 , 102 , 109 , 112 , 115 focusing instead on the other 2 aims ( P < .001). Vendor-measure studies were more likely to characterize EHR or clinical workflows , such as when EHR activities were performed during the day (68% vs 43% of articles, P = .016), while investigator-measure studies were more likely to characterize team dynamics , for example, using record coaccess to determine which clinicians routinely worked together (23% vs 3% of studies, P = .004).…”

Resident Physician Experience and Duration of Electronic Health Record Use

“…Because residents may not be the physician signing the note, this metric serves as a proxy measure of productivity and may not capture some patients who may be seen by resident physicians who do not sign the chart note. 33 Further, patients seen per day in an ambulatory setting may not be a good measure of resident productivity for a variety of reasons mentioned above, most specifically that resident physicians may not be in control of their workloads.…”

Design, Usability, and Acceptability of a Needs-Based, Automated Dashboard to Provide Individualized Patient-Care Data to Pediatric Residents

“…Prior studies have described barriers in developing dashboards for use by trainees, including challenges with patient attribution which can lead trainees to feel that metrics are not as meaningful. 5,[25][26][27][28] In our study, resident users seemed overall to understand the limitations of the dashboard, but similarly reported that some metrics were less meaningful on an individual basis due to patient attribution limitations. Resident comments indicated that they felt some metrics were more reflective of decisions made by the care team rather than an individual, which is consistent with findings of other studies regarding the challenges of creating resident-specific performance metrics.…”

“…Furthermore, our patient attribution was based exclusively on note writing, which may not perfectly reflect all patients cared for by a resident. 25,28 A resident may have participated meaningfully in the care of a patient, but if no note was written (for example, if care occurred overnight) then this would not be captured by the dashboard. Additionally, at our institution upper-level residents (PGY2 and above) typically sign fewer notes than interns on this rotation, which makes the results of some parts of the dashboard less relevant as residents advance in their training.…”