Background: Immune-checkpoint inhibitors may provide long-term survival benefits via a cured proportion, yet data are usually insufficient to prove this upon submission to health technology assessment bodies. Objective: We revisited the National Institute for Health and Care Excellence assessment of ipilimumab in melanoma (TA319). We used updated data from the pivotal trial to assess the accuracy of the extrapolation methods used and compared these to previously unused techniques to establish whether an alternative extrapolation may have provided more accurate survival projections. Methods: We compared projections from the piecewise survival model used in TA319 and those produced by alternative models (fit to trial data with minimum follow-up of 3 years) to a longer-term data cut (5-year follow-up). We also compared projections to external data to help assess validity. Alternative approaches considered were parametric, spline-based, mixture, and mixture-cure models. Results: Only the survival model used in TA319 and a mixture-cure model provided 5-year survival predictions close to those observed in the 5-year follow-up data set. Standard parametric, spline, and nonecurativemixture models substantially underestimated 5-year survival. Survival estimates from the TA319 model and the mixture-cure model diverge considerably after 5 years and remain unvalidated. Conclusions: In our case study, only models that incorporated an element of external information (through a cure fraction combined with background mortality rates or using registry data) provided accurate estimates of 5-year survival. Flexible models that were able to capture the complex hazard functions observed during the trial, but which did not incorporate external information, extrapolated poorly.

Objectives. In June 2011, the National Institute for Health and Care Excellence (NICE) Decision Support Unit published a Technical Support Document (TSD) providing recommendations on survival analysis for NICE technology appraisals (TAs). Survival analysis outputs are influential inputs into economic models estimating the cost-effectiveness of new cancer treatments. Hence, it is important that systematic and justifiable model selection approaches are used. This study investigates the extent to which the TSD recommendations have been followed since its publication. Methods. We reviewed NICE cancer TAs completed between July 2011 and July 2017. Information on survival analyses undertaken and associated critiques for overall survival (OS) and progression-free survival were extracted from the company submissions, Evidence Review Group (ERG) reports, and final appraisal determination documents. Results. Information was extracted from 58 TAs. Only 4 (7%) followed all TSD recommendations for OS outcomes. The vast majority (91%) compared a range of common parametric models and assessed their fit to the data (86%). Only a minority of TAs included an assessment of the shape of the hazard function (38%) or proportional hazards assumption (40%). Validation of the extrapolated portion of the survival function using external data was attempted in a minority of TAs (40%). Extrapolated survival functions were frequently criticized by ERGs (71%). Conclusions. Survival analysis within NICE TAs remains suboptimal, despite publication of the TSD. Model selection is not undertaken in a systematic way, resulting in inconsistencies between TAs. More attention needs to be given to assessing hazard functions and validation of extrapolated survival functions. Novel methods not described in the TSD have been used, particularly in the context of immuno-oncology, suggesting that an updated TSD may be of value.

Background The NHS emergency and urgent care system is under pressure as demand for services increases each year. NHS 111 is a telephone triage service designed to provide advice and signposting to appropriate services for people with urgent health-care problems. A new service, NHS 111 Online, has been introduced across England as a digital alternative that can be accessed using a website or a smartphone application. The effects and usefulness of this service are unknown. Objectives To explore the impact of NHS 111 Online on the related telephone service and urgent care system activity and the experiences of people who use those services. Design and methods A mixed-methods design of five related work packages comprising an evidence review; a quantitative before-and-after time series analysis of changes in call activity (18/38 sites); a descriptive comparison of telephone and online services with qualitative survey (telephone, n = 795; online, n = 3728) and interview (32 participants) studies of service users; a qualitative interview study (16 participants) of staff; and a cost–consequences analysis. Results The online service had little impact on the number of triaged calls to the NHS 111 telephone service. For every 1000 online contacts, triaged telephone calls increased by 1.3% (1.013, 95% confidence interval 0.996 to 1.029; p = 0.127). Recommendations to attend emergency and urgent care services increased between 6.7% and 4.2%. NHS 111 Online users were less satisfied than users of the telephone service (50% vs. 71%; p < 0.001), and less likely to recommend to others (57% vs. 69%; p < 0.001) and to report full compliance with the advice given (67.5% vs. 88%; p < 0.001). Online users were less likely to report contacting emergency services and more likely to report not making any contact with a health service (31% vs. 16%; p < 0.001) within 7 days of contact. Thirty-five per cent of online users reported that they did not want to use the telephone service, whereas others preferred its convenience and speed. NHS 111 telephone staff reported no discernible increase or decrease in their workload during the first year of operation of NHS 111 Online. If online and telephone services operate in parallel, then the annual costs will be higher unless ≥ 38% of telephone contacts move to online contacts. Conclusions There is some evidence that the new service has the potential to create new demand. The service has expanded significantly, so it is important to find ways of promoting the right balance in numbers of people who use the online service instead of the telephone service if it is to be effective. There is a clear need and preference by some people for an online service. Better information about when to use this service and improvements to questioning may encourage more uptake. Limitations The lack of control arm means that impact could have been an effect of other factors. This work took place during the early implementation phase, so findings may change as the service expands. Future work Further development of the online triage process to make it more ‘user friendly’ and to enable users to trust the advice given online could improve use and increase satisfaction. Better understanding of the characteristics of the telephone and online populations could help identify who is most likely to benefit and could improve information about when to use the service. Trial registration Current Controlled Trials ISRCTN51801112. Funding This project was funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research programme and will be published in full in Health Services and Delivery Research; Vol. 9, No. 21. See the NIHR Journals Library website for further project information.

Background. After failure of first-line antiretroviral therapy (ART) in the public sector, delayed or missed second-line ART switch is linked with poor outcomes in patients with advanced HIV. Setting:We investigated delayed or missed second-line ART switch following confirmed virologic failure in the largest private sector HIV cohort in Africa.Methods. We included HIV-infected adults with confirmed virologic failure after six months of non-nucleoside reverse-transcriptase inhibitor-based ART. We estimated the effect of timing of switch on the hazard of death using inverse probability of treatment weighting of marginal structural models. We adjusted for time-dependent confounding of CD4 count, viral load, and visit frequency. Results. 5748 patients (53% female) with confirmed virologic failure met inclusion criteria; the median age was 40 (interquartile range [IQR]: 35 -47), advanced HIV was present in 48% and the prior duration of NNRTI-based ART was 1083 days (IQR: 665-1770). Median time to confirmation of virologic failure and to second-line switch was 196 (IQR: 136-316) and 220 days (IQR: 65-542), respectively. Switching to second-line ART after confirmed failure compared to remaining on first-line ART reduced risk of subsequent death [aHR: 0.47 (95% CI: 0.36-0.63)].Compared to patients who experienced delayed switch, those switched immediately had a lower risk of death, regardless of CD4 cell count.

BackgroundAmbulance service quality measures have focused on response times and a small number of emergency conditions, such as cardiac arrest. These quality measures do not reflect the care for the wide range of problems that ambulance services respond to and the Prehospital Outcomes for Evidence Based Evaluation (PhOEBE) programme sought to address this.ObjectivesThe aim was to develop new ways of measuring the impact of ambulance service care by reviewing and synthesising literature on prehospital ambulance outcome measures and using consensus methods to identify measures for further development; creating a data set linking routinely collected ambulance service, hospital and mortality data; and using the linked data to explore the development of case-mix adjustment models to assess differences or changes in processes and outcomes resulting from ambulance service care.DesignA mixed-methods study using a systematic review and synthesis of performance and outcome measures reported in policy and research literature; qualitative interviews with ambulance service users; a three-stage consensus process to identify candidate indicators; the creation of a data set linking ambulance, hospital and mortality data; and statistical modelling of the linked data set to produce novel case-mix adjustment measures of ambulance service quality.SettingEast Midlands and Yorkshire, England.ParticipantsAmbulance services, patients, public, emergency care clinical academics, commissioners and policy-makers between 2011 and 2015.InterventionsNone.Main outcome measuresAmbulance performance and quality measures.Data sourcesAmbulance call-and-dispatch and electronic patient report forms, Hospital Episode Statistics, accident and emergency and inpatient data, and Office for National Statistics mortality data.ResultsSeventy-two candidate measures were generated from systematic reviews in four categories: (1) ambulance service operations (n = 14), (2) clinical management of patients (n = 20), (3) impact of care on patients (n = 9) and (4) time measures (n = 29). The most common operations measures were call triage accuracy; clinical management was adherence to care protocols, and for patient outcome it was survival measures. Excluding time measures, nine measures were highly prioritised by participants taking part in the consensus event, including measures relating to pain, patient experience, accuracy of dispatch decisions and patient safety. Twenty experts participated in two Delphi rounds to refine and prioritise measures and 20 measures scored ≥ 8/9 points, which indicated good consensus. Eighteen patient and public representatives attending a consensus workshop identified six measures as important: time to definitive care, response time, reduction in pain score, calls correctly prioritised to appropriate levels of response, proportion of patients with a specific condition who are treated in accordance with established guidelines, and survival to hospital discharge for treatable emergency conditions. From this we developed six new potential indicators using the linked data set, of which five were constructed using case-mix-adjusted predictive models: (1) mean change in pain score; (2) proportion of serious emergency conditions correctly identified at the time of the 999 call; (3) response time (unadjusted); (4) proportion of decisions to leave a patient at scene that were potentially inappropriate; (5) proportion of patients transported to the emergency department by 999 emergency ambulance who did not require treatment or investigation(s); and (6) proportion of ambulance patients with a serious emergency condition who survive to admission, and to 7 days post admission. Two indicators (pain score and response times) did not need case-mix adjustment. Among the four adjusted indicators, we found that accuracy of call triage was 61%, rate of potentially inappropriate decisions to leave at home was 5–10%, unnecessary transport to hospital was 1.7–19.2% and survival to hospital admission was 89.5–96.4% depending on Clinical Commissioning Group area. We were unable to complete a fourth objective to test the indicators in use because of delays in obtaining data. An economic analysis using indicators (4) and (5) showed that incorrect decisions resulted in higher costs.LimitationsCreation of a linked data set was complex and time-consuming and data quality was variable. Construction of the indicators was also complex and revealed the effects of other services on outcome, which limits comparisons between services.ConclusionsWe identified and prioritised, through consensus processes, a set of potential ambulance service quality measures that reflected preferences of services and users. Together, these encompass a broad range of domains relevant to the population using the emergency ambulance service. The quality measures can be used to compare ambulance services or regions or measure performance over time if there are improvements in mechanisms for linking data across services.Future workThe new measures can be used to assess different dimensions of ambulance service delivery but current data challenges prohibit routine use. There are opportunities to improve data linkage processes and to further develop, validate and simplify these measures.FundingThe National Institute for Health Research Programme Grants for Applied Research programme.