Examining organisational responses to performance-based financial incentive systems: a case study using NHS staff influenza vaccination rates from 2012/2013 to 2019/2020

Abstract

ObjectiveFinancial incentives are often applied to motivate desirable performance across organisations in healthcare systems. In the 2016/2017 financial year, the National Health Service (NHS) in England set a national performance-based incentive to increase uptake of the influenza vaccination among frontline staff. Since then, the threshold levels needed for hospital trusts to achieve the incentive (ie, the targets) have ranged from 70% to 80%. The present study examines the impact of this financial incentive across eight vaccination seasons.DesignA retrospective observational study examining routinely recorded rates of influenza vaccination among staff in all acute NHS hospital trusts across eight vaccination seasons (2012/2013–2019/2020). The number of trusts included varied per year, from 127 to 137, due to organisational changes. McCrary’s density test is conducted to determine if the number of hospital trusts narrowly achieving the target by the end of each season is higher than would be expected in the absence of any responsiveness to the target. We refer to this bunching above the target threshold as a ‘threshold effect’.ResultsIn the years before a national incentive was set, 9%–31% of NHS Trusts reported achieving the target, compared with 43%–74% in the 4 years after. Threshold effects did not emerge before the national incentive for payment was set; however, since then, threshold effects have appeared every year. Some trusts report narrowly achieving the target each year, both as the target rises and falls. Threshold effects were not apparent at targets for partial payments.ConclusionsWe provide compelling evidence that performance-based financial incentives produced threshold effects. Policymakers who set such incentives are encouraged to track threshold effects since they contain information on how organisations are responding to an incentive, what enquiries they may wish to make, how the incentive may be improved and what unintended effects it may be having.

Unintended consequences of the 18-week referral to treatment standard in NHS England: a threshold analysis

“…There are two types of behaviour by hospital trusts that may underlie threshold effects. 13 The first behaviour concerns how hospital trusts perform in relation to achieving the target. Some hospital trusts near the 92% target will take action to clear the target, while those further from the target do not take such action (or are unsuccessful in doing so).…”

“…The NHS 4-hour A&E service standard 9 and the Quality and Outcome Framework requirement to measure blood pressure in primary care 10 both recorded spikes at their respective performance thresholds. However, they did not use formal statistical methods to check for discontinuity at the threshold targets such as those developed by McCrary 11 and Cattaneo et al 12 An exception is a study examining NHS staff influenza vaccination rates 13 that used McCrary's test to track the effects of the target as it was changed by the government from year to year.…”

“…However, they did not use formal statistical methods to check for discontinuity at the threshold targets such as those developed by McCrary 11 and Cattaneo et al . 12 An exception is a study examining NHS staff influenza vaccination rates 13 that used McCrary’s test to track the effects of the target as it was changed by the government from year to year.…”

“…(8)(9)(10)(11)(12)(13)(14)(15)(16)(17)(18)(19)(20)(21)(22)(23) on May 25, 2024 by guest. Protected by copyright.http://qualitysafety.bmj.com/ BMJ Qual Saf: first published as 10.1136/bmjqs-2023-015949 on 5 September 2023.…”

Pay-for-performance incentives for health worker vaccination: looking under the hood

“…Prior efforts to increase influenza vaccination uptake among healthcare workers at the health system level are an important part of that evidence. In this issue of BMJ Quality & Safety , Liaqat and colleagues12 analyse health system responses to performance-based financial incentives for healthcare worker influenza vaccination that were introduced by England’s National Health Service (NHS) in 2016/2017. Using data from eight vaccination seasons that span the period before and after the introduction of these incentives, the authors demonstrate in a convincing manner that the pay-for-performance scheme led to an increase in influenza vaccination rates among frontline staff.…”