Review article: What makes a good healthcare quality indicator? A systematic review and validation study

Abstract

Indicators measuring aspects of performance to assess quality of care are often chosen arbitrarily. The present study aimed to determine what should be considered when selecting healthcare quality indicators, particularly focusing on the application to emergency medicine. Structured searches of electronic databases were supplemented by website searches of quality of care and benchmarking organisations, citation searches and discussions with experts. Candidate attributes of 'good' healthcare indicators were extracted independently by two authors. The validity of each attribute was independently assessed by 16 experts in quality of care and emergency medicine. Valid and reliable attributes were included in a critical appraisal tool for healthcare quality indicators, which was piloted by emergency medicine specialists. Twenty-three attributes were identified, and all were rated moderate to extremely important by an expert panel. The reliability was high: alpha = 0.98. Twelve existing tools explicitly stated a median (range) of 14 (8-17) attributes. A critical appraisal tool incorporating all the attributes was developed. This was piloted by four emergency medicine specialists who were asked to appraise and rank a set of six candidate indicators. Although using the tool took more time than implicit gestalt decision making: median (interquartile range) 190 (43-352) min versus 17.5 (3-34) min, their rankings changed after using the tool. To inform the appraisal of quality improvement indicators for emergency medicine, a comprehensive list of indicator attributes was identified, validated, developed into a tool and piloted. Although expert consensus is still required, this tool provides an explicit basis for discussions around indicator selection.

Promoting further use of palliative care in cancer care centers

“…Indicators though need to go beyond consensus have face validity, also discriminate between objective good and bad outcomes whether patient-centered or within economic domains, and be meaningful (not just statistically significant). Indicators should be clinically relevant, achievable, valid, and applicable as judged by important parties [31,35,36] The Appraisal of Indicators through Research and Evaluation is a standard guide for those who wish to develop indicators [37]. There are eight palliative-care domains concerning quality of care (Tables 2 and 3) [38,39].…”

Towards a best measure of emergency department crowding: Lessons from current Australasian practice

“…To date, there have been no studies exploring how ED crowding is measured in Australasia. When selecting quality indicators in healthcare, it is important to know whether the indicator is currently used, which attests its face validity and ability to be measured easily …”

“…When selecting quality indicators in healthcare, it is important to know whether the indicator is currently used, which attests its face validity and ability to be measured easily. 15 The primary aim of the study was to determine which ED crowding metrics are used in Australia and New Zealand and the purposes for which these metrics were used. This would inform the discussion around • ED crowding is still a common and important problem in Australia and New Zealand.…”

Is a national time target for emergency department stay associated with changes in the quality of care for acute asthma? A multicentre pre‐intervention post‐intervention study

“…Prior studies evaluating the effects of ED time targets have largely drawn on routinely available data and focused on lengths of stay, admission rates, numbers of investigations, and overall mortality rather than clinical quality of care indicators . Quality indicators in health care are considered most useful if they focus on high cost or high volume conditions and address equity across a wide range of demographic groups . Acute asthma is a common and potentially life threatening condition that results in approximately 4500 hospital admissions annually in NZ .…”