Assessing hospital performance indicators. What dimensions? Evidence from an umbrella review

Carini, E; Gabutti, Irene; Frisicale, Emanuela Maria; Pilla, Andrea Di; Pezzullo, Angelo Maria; Waure, Chiara de; Cicchetti, Americo; Boccia, Stefania; Specchia, Maria Lucia

doi:10.1186/s12913-020-05879-y

Cited by 44 publications

(29 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The resulting dimensions from this project were clinical effectiveness, efficiency, HCW orientation, responsive governance, safety, and patient-centeredness. However, studies have shown that there are still some gaps in this model and issues concerning the dimensions investigated [ 13 , 14 ]. Additionally, the Organization for Economic Co-operation and Development (OECD) launched the Health Care Quality Indicator (HCQI) project in 2006; it aimed to develop key performance indicators (KPIs) to compare quality in health care at the international level and achieve international benchmarking.…”

Section: Introductionmentioning

confidence: 99%

A systematic review: the dimensions to evaluate health care performance and an implication during the pandemic

Amer

Hammoud

Khatatbeh

et al. 2022

BMC Health Serv Res

View full text Add to dashboard Cite

Background The balanced scorecard (BSC) has been implemented to evaluate the performance of health care organizations (HCOs). BSC proved to be effective in improving financial performance and patient satisfaction. Aim This systematic review aims to identify all the perspectives, dimensions, and KPIs that are vital and most frequently used by health care managers in BSC implementations. Methods This systematic review adheres to PRISMA guidelines. The PubMed, Embase, Cochrane, and Google Scholar databases and Google search engine were inspected to find all implementations of BSC at HCO. The risk of bias was assessed using the nonrandomized intervention studies (ROBINS-I) tool to evaluate the quality of observational and quasi-experimental studies and the Cochrane (RoB 2) tool for randomized controlled trials (RCTs). Results There were 33 eligible studies, of which we identified 36 BSC implementations. The categorization and regrouping of the 797 KPIs resulted in 45 subdimensions. The reassembly of these subdimensions resulted in 13 major dimensions: financial, efficiency and effectiveness, availability and quality of supplies and services, managerial tasks, health care workers' (HCWs) scientific development error-free and safety, time, HCW-centeredness, patient-centeredness, technology, and information systems, community care and reputation, HCO building, and communication. On the other hand, this review detected that BSC design modification to include external and managerial perspectives was necessary for many BSC implementations. Conclusion This review solves the KPI categorization dilemma. It also guides researchers and health care managers in choosing dimensions for future BSC implementations and performance evaluations in general. Consequently, dimension uniformity will improve the data sharing and comparability among studies. Additionally, despite the pandemic negatively influencing many dimensions, the researchers observed a lack of comprehensive HCO performance evaluations. In the same vein, although some resulting dimensions were assessed separately during the pandemic, other dimensions still lack investigation. Last, BSC dimensions may play an essential role in tackling the COVID-19 pandemic. However, further research is required to investigate the BSC implementation effect in mitigating the pandemic consequences on HCO.

show abstract

Section: Introductionmentioning

confidence: 99%

A systematic review: the dimensions to evaluate health care performance and an implication during the pandemic

Amer

Hammoud

Khatatbeh

et al. 2022

BMC Health Serv Res

View full text Add to dashboard Cite

show abstract

“…The resulting dimensions from this project were clinical effectiveness, efficiency, HCW orientation, responsive governance, safety, and patientcenteredness. However, studies have shown that there are still some gaps in this model and issues concerning the dimensions investigated [13,14]. Additionally, the Organization for Economic Co-operation and Development (OECD) launched the Health Care Quality Indicator (HCQI) project in 2006; it aimed to develop key performance indicators (KPIs) to compare quality in health care at the international level and achieve international benchmarking.…”

Section: Introductionmentioning

confidence: 99%

A Systematic Review: The Dimensions utilized in the Performance Evaluation of Healthcare- An Implication during the COVID-19 Pandemic

Amer

Hammoud

Khatatbeh

et al. 2021

Preprint

View full text Add to dashboard Cite

This systematic review aims to identify the Key Performance Indicators (KPIs) which are the most frequently used and important in the Performance Evaluation (PE) of Health Care Organizations (HCOs). Also, it aimed to analyze the resulted Balanced Scorecard (BSC) dimensions during the Coronavirus Disease 2019 (COVID-19) era. This systematic review was prepared according to PRISMA guidelines. PubMed, Embase, Cochrane, and Google Scholar databases, as well as Google search engine, were inspected to find all implementations of BSC at HCOs until 20 September 2020. The Risk of Bias (RoB) was assessed for each included article. The eligible studies were 33, in which 36 BSC implementations were identified. The categorization and re-grouping of the 797 KPIs resulted in 13 major-dimensions: The financial, efficiency and effectiveness, availability and quality of supplies and services, managerial tasks, Health Care Workers (HCWs) scientific development error-free and safety, the time, HCW-centeredness, patient-centeredness, technology and information system, community care and reputation, HCO building, and the communication. Under them, 45 sub-dimension were identified. The high RoB in most implementations was due to the selection bias. None of the 36 BSC implementations was utilized during the COVID-19 pandemic. This review solves the dilemma of categorization differences among BSC implementations, which in turn will improve the data sharing and comparability among studies. A modification for the BSC design to include external and managerial perspectives is required. However, a lack of a comprehensive PE for HCOs during the pandemic was observed, so the resulted dimensions and KPIs are recommended for planning, monitoring, evaluation, and continuous improvement of HCOs. Many dimensions were negatively influenced by the pandemic. However, some dimensions still need further investigation. BSC dimensions have an essential role in tackling the COVID-19 pandemic and will help the health care managers to mitigate its consequences on HCOs.

show abstract

“…Fourth, we assessed only one quality outcome, mortality. Hospital quality is a complex and multi-faceted construct [ 21 ] which cannot be summarized by hospital mortality alone, or by any single metric. However, mortality is a key performance indicator, and the methods of HS-TM and regression can be applied to other outcomes such that the findings of improved interpretability and credibility are not necessarily specific to mortality only.…”

Section: Discussionmentioning

confidence: 99%

Interpretability, credibility, and usability of hospital-specific template matching versus regression-based hospital performance assessments; a multiple methods study

McGrath

Takamine

Hogan

et al. 2022

BMC Health Serv Res

View full text Add to dashboard Cite

Background Hospital-specific template matching (HS-TM) is a newer method of hospital performance assessment. Objective To assess the interpretability, credibility, and usability of HS-TM-based vs. regression-based performance assessments. Research design We surveyed hospital leaders (January-May 2021) and completed follow-up semi-structured interviews. Surveys included four hypothetical performance assessment vignettes, with method (HS-TM, regression) and hospital mortality randomized. Subjects Nationwide Veterans Affairs Chiefs of Staff, Medicine, and Hospital Medicine. Measures Correct interpretation; self-rated confidence in interpretation; and self-rated trust in assessment (via survey). Concerns about credibility and main uses (via thematic analysis of interview transcripts). Results In total, 84 participants completed 295 survey vignettes. Respondents correctly interpreted 81.8% HS-TM vs. 56.5% regression assessments, p < 0.001. Respondents “trusted the results” for 70.9% HS-TM vs. 58.2% regression assessments, p = 0.03. Nine concerns about credibility were identified: inadequate capture of case-mix and/or illness severity; inability to account for specialized programs (e.g., transplant center); comparison to geographically disparate hospitals; equating mortality with quality; lack of criterion standards; low power; comparison to dissimilar hospitals; generation of rankings; and lack of transparency. Five concerns were equally relevant to both methods, one more pertinent to HS-TM, and three more pertinent to regression. Assessments were mainly used to trigger further quality evaluation (a “check oil light”) and motivate behavior change. Conclusions HS-TM-based performance assessments were more interpretable and more credible to VA hospital leaders than regression-based assessments. However, leaders had a similar set of concerns related to credibility for both methods and felt both were best used as a screen for further evaluation.

show abstract

Assessing hospital performance indicators. What dimensions? Evidence from an umbrella review

Cited by 44 publications

References 20 publications

A systematic review: the dimensions to evaluate health care performance and an implication during the pandemic

A systematic review: the dimensions to evaluate health care performance and an implication during the pandemic

A Systematic Review: The Dimensions utilized in the Performance Evaluation of Healthcare- An Implication during the COVID-19 Pandemic

Interpretability, credibility, and usability of hospital-specific template matching versus regression-based hospital performance assessments; a multiple methods study

Contact Info

Product

Resources

About