Design, Usability, and Acceptability of a Needs-Based, Automated Dashboard to Provide Individualized Patient-Care Data to Pediatric Residents

Abstract

Background and Objectives Pediatric residency programs are required by the Accreditation Council for Graduate Medical Education to provide residents with patient-care and quality metrics to facilitate self-identification of knowledge gaps to prioritize improvement efforts. Trainees are interested in receiving this data, but this is a largely unmet need. Our objectives were to (1) design and implement an automated dashboard providing individualized data to residents, and (2) examine the usability and acceptability of the dashboard among pediatric residents. Methods We developed a dashboard containing individualized patient-care data for pediatric residents with emphasis on needs identified by residents and residency leadership. To build the dashboard, we created a connection from a clinical data warehouse to data visualization software. We allocated patients to residents based on note authorship and created individualized reports with masked identities that preserved anonymity. After development, we conducted usability and acceptability testing with 11 resident users utilizing a mixed-methods approach. We conducted interviews and anonymous surveys which evaluated technical features of the application, ease of use, as well as users' attitudes toward using the dashboard. Categories and subcategories from usability interviews were identified using a content analysis approach. Results Our dashboard provides individualized metrics including diagnosis exposure counts, procedure counts, efficiency metrics, and quality metrics. In content analysis of the usability testing interviews, the most frequently mentioned use of the dashboard was to aid a resident's self-directed learning. Residents had few concerns about the dashboard overall. Surveyed residents found the dashboard easy to use and expressed intention to use the dashboard in the future. Conclusion Automated dashboards may be a solution to the current challenge of providing trainees with individualized patient-care data. Our usability testing revealed that residents found our dashboard to be useful and that they intended to use this tool to facilitate development of self-directed learning plans.

Uses of Electronic Health Record Data to Measure the Clinical Learning Environment of Graduate Medical Education Trainees: A Systematic Review

“…By contrast, in our review, a handful of scalable technologies (measures, attribution, and visualization) supported trainee experiential learning in myriad ways. Linking these technologies in an experiential learning cycle, as has been described by 2 research teams, 117,118 yields what we term a digital learning cycle (DLC): an experiential learning cycle driven by EHR data and scalable processes (Figure 2).…”

Use of a Medication Safety Audit and Feedback Tool in the Emergency Department Is Affected by Prescribing Characteristics

“…1 The EQUIPPED dashboard landing page, which contains the following core audit and feedback components: key performance indicators (A), peer-to-peer benchmarking (B), individual patient or encounter drill down (C), alternative medications (D), and longitudinal performance tracking (E). 16 access). This outcome was chosen to measure which ED providers attempted to initially engage with the EQUIPPED dashboard once made available to them.…”

“… 9 10 11 Poor dashboard utilization and lack of engagement may be responsible for the inconsistencies observed in the evidence base. 12 The integration of human factors principles during dashboard development followed by implementation of formal usability study can lead to improved dashboard satisfaction, 13 14 15 16 17 but beyond age, 18 there is little evidence on what factors may initially drive dashboard engagement once made available for routine use. Developing a better understanding of the facilitators and barriers to clinical dashboard utilization in different clinical settings would add further insight into dashboard orientation strategies and work practice implementation.…”

“… The EQUIPPED dashboard landing page, which contains the following core audit and feedback components: key performance indicators ( A ), peer-to-peer benchmarking ( B ), individual patient or encounter drill down ( C ), alternative medications ( D ), and longitudinal performance tracking ( E ). 16 …”