Prioritizing measures to assess performance of drug treatment services: a Delphi process with funders, treatment providers and service‐users

Abstract

Background and aim While many studies have examined outcome measurement as part of clinical trials and routine outcome collection at the person‐level in alcohol and other drug (AOD) treatment services, there has been limited attention to measures required to assess performance at the service‐level. In Australia, non‐government services are primarily funded by government using public funds; however, there is no standardized approach to performance measurement. This study sought to establish a finite list of performance measures that represented consensus between funders, treatment providers and service‐users. Method A three‐round Delphi process was undertaken with (i) funders of treatment (n = 10), (ii) treatment providers (n = 10) and (iii) treatment service‐users (n = 10). Participants were asked to rate a range of measures on a 10‐point Likert scale on how important they were to be included in contracts with funders. Measures with a median score > 7 and agreement among participants above 70% were the criteria for inclusion in the final set of measures. Qualitative data in the form of text responses provided by participants for their ratings in rounds 1 and 2 were also analysed. Results Participants rated 93 measures in round 1, which reduced to 78 measures in round 2 and 32 measures in round 3. Fifteen service‐level measures and two system‐level measures met criteria for inclusion in the final set of performance measures. The final set of measures cover a range of measurement types: outcomes (n = 5), access (n = 3), structural (n = 3), experience (n = 2), input (n = 2), process (n = 1) and output (n = 1). Conclusion In Australia, performance measures for alcohol and other drug treatment services that represent a consensus among service‐users, providers and funders focus upon demonstrating accountability for public funds, improving services and communicating key measures of success to future service‐users and the broader community.

A social network analysis approach to assess COVID19-related disruption to substance use treatment and informal social interactions among people who use drugs in Scotland

“…In a recent Delphi study to identify performance measures for addiction treatment services, service users placed a higher priority (than funders or providers) on the extent to which service users are linked up to other services [18]. In a context where services compete against each other for funding, there is a disincentive to make one service's performance metric dependent on a third party.…”