Usability Testing a Potentially Inappropriate Medication Dashboard: A Core Component of the Dashboard Development Process

Lagha, Regina Richter; Burningham, Zachary; Sauer, Brian C.; Leng, Jianwei; Peters, Celena B.; Huynh, Tina; Patel, Seraphim; Halwani, Ahmad; Kramer, B. Josea

doi:10.1055/s-0040-1714693

Cited by 22 publications

(25 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The use of site-specific data would encourage participants to scrutinize visualizations more thoroughly to confirm that they are configured as required for care quality monitoring before installation. Other studies have highlighted similar messages about how the interpretability of information needs greater consideration in design processes alongside usability and ease of use [ 35 ]. Our study highlighted that site champions need to have confidence that metric configurations are aligned with service expectations, to support interpretation by site staff, and to lead its use in quality improvement efforts.…”

Section: Discussionmentioning

confidence: 98%

Analysis of a Web-Based Dashboard to Support the Use of National Audit Data in Quality Improvement: Realist Evaluation

Alvarado¹,

McVey²,

Greenhalgh³

et al. 2021

J Med Internet Res

View full text Add to dashboard Cite

Background Dashboards can support data-driven quality improvements in health care. They visualize data in ways intended to ease cognitive load and support data comprehension, but how they are best integrated into working practices needs further investigation. Objective This paper reports the findings of a realist evaluation of a web-based quality dashboard (QualDash) developed to support the use of national audit data in quality improvement. Methods QualDash was co-designed with data users and installed in 8 clinical services (3 pediatric intensive care units and 5 cardiology services) across 5 health care organizations (sites A-E) in England between July and December 2019. Champions were identified to support adoption. Data to evaluate QualDash were collected between July 2019 and August 2021 and consisted of 148.5 hours of observations including hospital wards and clinical governance meetings, log files that captured the extent of use of QualDash over 12 months, and a questionnaire designed to assess the dashboard’s perceived usefulness and ease of use. Guided by the principles of realist evaluation, data were analyzed to understand how, why, and in what circumstances QualDash supported the use of national audit data in quality improvement. Results The observations revealed that variation across sites in the amount and type of resources available to support data use, alongside staff interactions with QualDash, shaped its use and impact. Sites resourced with skilled audit support staff and established reporting systems (sites A and C) continued to use existing processes to report data. A number of constraints influenced use of QualDash in these sites including that some dashboard metrics were not configured in line with user expectations and staff were not fully aware how QualDash could be used to facilitate their work. In less well-resourced services, QualDash automated parts of their reporting process, streamlining the work of audit support staff (site B), and, in some cases, highlighted issues with data completeness that the service worked to address (site E). Questionnaire responses received from 23 participants indicated that QualDash was perceived as useful and easy to use despite its variable use in practice. Conclusions Web-based dashboards have the potential to support data-driven improvement, providing access to visualizations that can help users address key questions about care quality. Findings from this study point to ways in which dashboard design might be improved to optimize use and impact in different contexts; this includes using data meaningful to stakeholders in the co-design process and actively engaging staff knowledgeable about current data use and routines in the scrutiny of the dashboard metrics and functions. In addition, consideration should be given to the processes of data collection and upload that underpin the quality of the data visualized and consequently its potential to stimulate quality improvement. International Registered Report Identifier (IRRID) RR2-10.1136/bmjopen-2019-033208

show abstract

Section: Discussionmentioning

confidence: 98%

Analysis of a Web-Based Dashboard to Support the Use of National Audit Data in Quality Improvement: Realist Evaluation

Alvarado¹,

McVey²,

Greenhalgh³

et al. 2021

J Med Internet Res

View full text Add to dashboard Cite

show abstract

“…These customized data displays also focused on a diverse set of application areas, including integrated dashboards [ 10 , 61 , 124 ], critical care displays [ 96 ], opioid management [ 123 ], plan of care tools [ 125 ], and patient-focused communication [ 11 , 55 , 98 ]. A smaller number of studies addressed non-EHR integrated information systems [ 9 , 65 , 68 , 83 , 112 , 120 ], and EHR training design [ 46 , 59 , 135 ].…”

Section: Resultsmentioning

confidence: 99%

Human Factors and Organizational Issues in Health Informatics: Innovations and Opportunities

Marquard

2021

Yearb Med Inform

View full text Add to dashboard Cite

Objective: Human factors and ergonomics (HF/E) frameworks and methods are becoming embedded in the health informatics community. There is now broad recognition that health informatics tools must account for the diverse needs, characteristics, and abilities of end users, as well as their context of use. The objective of this review is to synthesize the current nature and scope of HF/E integration into the health informatics community. Methods: Because the focus of this synthesis is on understanding the current integration of the HF/E and health informatics research communities, we manually reviewed all manuscripts published in primary HF/E and health informatics journals during 2020. Results: HF/E-focused health informatics studies included in this synthesis focused heavily on EHR customizations, specifically clinical decision support customizations and customized data displays, and on mobile health innovations. While HF/E methods aimed to jointly improve end user safety, performance, and satisfaction, most HF/E-focused health informatics studies measured only end user satisfaction. Conclusion: HF/E-focused health informatics researchers need to identify and communicate methodological standards specific to health informatics, to better synthesize findings across resource intensive HF/E-focused health informatics studies. Important gaps in the HF/E design and evaluation process should be addressed in future work, including support for technology development platforms and training programs so that health informatics designers are as diverse as end users.

show abstract

“…Involving end users and inviting qualitative feedback are associated with efficient and effective dashboards tailored to the population that will use them. 49 50 51 The design team and stakeholders included people with different kinds of expertise, including informatics experts, maternal and child health subject matter experts, scientific editors, and government employees that organize community health programs to ensure the tool would be accessible across users.…”

Section: Discussionmentioning

confidence: 99%

Developing Dashboards to Address Children's Health Disparities in Ohio

et al. 2022

View full text Add to dashboard Cite

Objectives Social determinants of health (SDoH) can be measured at the geographic level to convey information about neighborhood deprivation. The Ohio Children's Opportunity Index (OCOI) is a composite area-level opportunity index comprised of eight health domains. Our research team has documented the design, development, and use cases of a dashboard solution to visualize OCOI. Methods The OCOI is a multidomain index spanning the following eight domains: (1) family stability, (2) infant health, (3) children's health, (4) access, (5) education, (6) housing, (7) environment, and (8) criminal justice. Information on these eight domains is derived from the American Community Survey and other administrative datasets. Our team used the Tableau Desktop visualization software and applied a user-centered design approach to developing the two OCOI dashboards—main OCOI dashboard and OCOI-race dashboard. We also performed convergence analysis to visualize the census tracts where different health indicators simultaneously exist at their worst levels. Results The OCOI dashboard has multiple, interactive components as follows: a choropleth map of Ohio displaying OCOI scores for a specific census tract, graphs presenting OCOI or domain scores to compare relative positions for tracts, and a sortable table to visualize scores for specific county and census tracts. A case study using the two dashboards for convergence analysis revealed census tracts in neighborhoods with low infant health scores and a high proportion of minority population. Conclusion The OCOI dashboards could assist health care leaders in making decisions that enhance health care delivery and policy decision-making regarding children's health particularly in areas where multiple health indicators exist at their worst levels.

show abstract

Usability Testing a Potentially Inappropriate Medication Dashboard: A Core Component of the Dashboard Development Process

Cited by 22 publications

References 27 publications

Analysis of a Web-Based Dashboard to Support the Use of National Audit Data in Quality Improvement: Realist Evaluation

Analysis of a Web-Based Dashboard to Support the Use of National Audit Data in Quality Improvement: Realist Evaluation

Human Factors and Organizational Issues in Health Informatics: Innovations and Opportunities

Developing Dashboards to Address Children's Health Disparities in Ohio

Contact Info

Product

Resources

About