Trainee and Program Director Perspectives on Meaningful Patient Attribution and Clinical Outcomes Data

Abstract

Background The Accreditation Council for Graduate Medical Education specifies that trainees must receive clinical outcomes and quality benchmark data at specific levels related to institutional patient populations. Program directors (PDs) are challenged to identify meaningful data and provide them in formats acceptable to trainees. Objective We sought to understand what types of patients, data/metrics, and data delivery systems trainees and PDs prefer for supplying trainees with clinical outcomes data. Methods Trainees (n = 21) and PDs (n = 12) from multiple specialties participated in focus groups during academic year 2017–2018. They described key themes for providing clinical outcomes data to trainees. Results Trainees and PDs differed in how they identified patients for clinical outcomes data for trainees. Trainees were interested in encounters where they felt a sense of responsibility or had autonomy/independent decision-making opportunities, continuity, or learned something new; PDs used broader criteria including all patients cared for by their trainees. Both groups thought trainees should be given trainee-level metrics and consistently highlighted the importance of comparison to peers and/or benchmarks. Both groups found value in “push” and “pull” data systems, although trainees wanted both, while PDs wanted one or the other. Both groups agreed that trainees should review data with specific faculty. Trainees expressed concern about being judged based on their patients' clinical outcomes. Conclusions Trainee and PD perspectives on which patients they would like outcomes data for differed, but they overlapped for types of metrics, formats, and review processes for the data.

Design, Usability, and Acceptability of a Needs-Based, Automated Dashboard to Provide Individualized Patient-Care Data to Pediatric Residents

“…Prior studies have described barriers in developing dashboards for use by trainees, including challenges with patient attribution which can lead trainees to feel that metrics are not as meaningful. 5 25 26 27 28 In our study, resident users seemed overall to understand the limitations of the dashboard, but similarly reported that some metrics were less meaningful on an individual basis due to patient attribution limitations. Resident comments indicated that they felt some metrics were more reflective of decisions made by the care team rather than an individual, which is consistent with findings of other studies regarding the challenges of creating resident-specific performance metrics.…”

“…Resident comments indicated that they felt some metrics were more reflective of decisions made by the care team rather than an individual, which is consistent with findings of other studies regarding the challenges of creating resident-specific performance metrics. 5 26 29 Another study has prioritized a list of resident-specific quality metrics which could mitigate this issue, but these metrics primarily focused on content captured within resident documentation (e.g., work of breathing or response to therapy documentation). 30 While these metrics would be very specific to the work of an individual resident, these data are very challenging to integrate into an automated tool without sophisticated natural language processing, so we were not able to include these metrics in this iteration of our dashboard.…”

“…4 Previous work has shown that trainees are interested in receiving patient-care data in the form of individualized case logs and other rotation-specific quality metrics. 5,6 Despite accrediting body requirements, increasing prevalence of institutional quality dashboards, and trainee desire for personalized performance data, only a few studies exist among procedural and radiological specialties which discuss dashboard development for automated case-logging and tracking. [6][7][8][9] Even fewer studies describe the creation of dashboards that provide quality metrics for trainees.…”

A Mixed-Methods Program Evaluation of a Self-directed Learning Panel Management Curriculum in an Internal Medicine Residency Clinic

“…19,28 Focus groups revealed residents felt empowered and appreciated this autonomy, but they found it problematic when goals were not relevant to patient visits. Previous work has suggested using resident-sensitive metrics, or metrics that are actionable and appropriate for Residents who answered strongly agree or agree were considered to "agree" (Likert scale 1-5: strongly disagree, disagree, neutral, agree, strongly agree) resident panel populations, and align with educational 19,[29][30][31] Continuous review of whether metrics are appropriate for residents may be needed. It is also plausible that more time may increase the likelihood that residents would have pertinent patient visits, but future work is needed to address how to better align learner needs with patient care.…”

Pursuing Excellence: Integrating Clinical Learning Environment Staff and Learners Into the Pursuit of Quality, Safety, Equity, and Value

“…10 Two key elements to consider when providing outcomes data are the extent to which the data are timely and actionable. 11 In many large programs, residents rotate between services, units, and even health systems on a monthly basis. If residents are to change their own behavior based on data, they must receive data while they are still on the rotation.…”