Can you measure performance?

Abstract

The National Health Service collects a vast amount of information on a routine basis, but much of it is unused. For years any attempts to use such information to evaluate performance has been criticised by the medical profession. The fact that annual hospital returns fail to distinguish between discharge and death, or that a hospital activity analysis print out sometimes presents the number of women patients suffering from diseases of the male genital organ, are two of many examples that serve to undermine confidence in the statistical information produced by the NHS.Reservations about using routinely collected data can be divided broadly into three areas: technical, conceptual, and emotional. Firstly, although we might hope that data would display certain technical characteristics like accuracy, completeness, relevance, and timeliness, they rarely do. Information collection can be a chore that is frequently delegated to the most junior staff, with adverse effects on its accuracy and completeness. A vicious circle develops in which information is not used because it is inaccurate and inaccurate because it is not used. The information that is presented invariably comes in an unattractive manner, with rows of figures rather like a railway timetable. Furthermore, the NHS tends to gather together information on a national basis in an aggregated form thus making district by district comparisons virtually impossible.Secondly, the concept of examining the performance of any health service is traditionally based on using indicators of input, process, outcome, need, demand, and environmental influences.' Our understanding of relationships between these six dimensions is limited. To what extent is case fatality (outcome) influenced by the level of staffing (input), length of stay (process), incidence and prevalence of the condition (need), the patients' expectation and knowledge (demand), and their socioeconomic circumstances (environmental influences) ? Our attempts to answer this type of question tend to polarise around two sorts of study. There are those that are detailed but include small numbers of patients-for example, randomised controlled trials-and those that generalise about morbidity using national census data. We can say with confidence that Charnley hip prostheses may be successfully implanted in patients suffering from arthritis and that for certain conditions older patients will stay in hospital longer than younger patients. What we do not know is the extent to which the traumatic and orthopaedic services in a district are acceptable and whether, given differences Collecting and processing dataWe started our study of routine data by looking at separate mental illness and mental handicap hospitals of over 100 beds in England. Later we extended the study to examine 34 selected specialties on a district basis. From the outset we accepted that a statistical picture would never precisely portray how a service or hospital performs, but, acknowledging that deficiency, we took the following approach.We examined ...

Using routine intelligence to target inspection of healthcare providers in England

“…This draws on earlier work such as that of Yates13 and others to look at organisational performance using multiple process measures, and the common scoring approach (modified Z scores) has parallels to earlier work looking at combining multiple metrics 7 14 15. The idea that information can be used to prompt further queries in a staged process was described by Lilford et al 16.…”

Methods and consequences of changes in use of episiotomy

“…13 (2 8) 18 (2 8) 16 (1-7) 16 ( 2 4)Complete tear of anal sphincter 2 (0 3) 6 (1 0) 9 (1 9) 9 (1-4) 8 (0 8) 6 (0 9)…”

“…Different ways of presenting statistical information about management in clinical settings exist.7 Graphical profiles may be used to present maternity unit statistics,8 9 and they provide means of comparing units and perhaps also individual subjects-for example, midwives-within units. Whether feedback of information by means of such profiles has any effect on decision making is, however, unknown."…”

Avoidable Death as a Measure of Quality

“…performance indicators (e.g. Yates [8]), and, in the UK, they are now being introduced by the Department of Health. Obviously, despite the evident usefulness of this measure of quality, and the previous paucity of work in this area, there has been a number of criticisms levelled at this method of assessing outcome, which ought to be addressed.…”