Development and assessment of a machine learning tool for predicting emergency admission in Scotland

Liley, James; Bohner, Gergő; Emerson, Samuel R.; Mateen, Bilal A.; Borland, Katie; Carr, David L.; Heald, Scott; Oduro, Samuel D.; Ireland, Jill; Moffat, Keith; Porteous, Rachel; Riddell, Stephen; Cunningham, Nathan; Holmes, Chris; Payne, Katrina; Vollmer, Sebastian J.; Vallejos, Catalina A.; Aslett, Louis J. M.

doi:10.1101/2021.08.06.21261593

Cited by 6 publications

(17 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“… 27 Where sensitivity and specificity were estimated from figure 2 (a) ROC plot. 27 Future algorithm research would benefit from application of consistent definitions so that developed algorithms may be tested and applied within different healthcare contexts (rural, remote and metropolitan) and countries.…”

Section: Discussionmentioning

confidence: 99%

“…In addition, lower socioeconomic status and lack of social support was predictive of unplanned readmissions, which was in agreement with SPARRA who used the Scottish Index of Multiple Deprivation using SHAP scores. 21 Both QAdmission 26 and SPARRA 27 found pathology and medication history to be an important feature for prediction of readmission, which would explain their higher performance. These data were deliberately left out so that other jurisdictions could build our model.…”

Section: Discussionmentioning

confidence: 99%

“…Our model is compared with other case-finding algorithms that predict one or more unplanned admission over a year using weighted scores 25 26 and machine learning. 27 Even though this was not the focus of the research, it provided some reasurrance of the methodology.…”

Section: Methodsmentioning

confidence: 99%

See 2 more Smart Citations

Novel machine learning model for predicting multiple unplanned hospitalisations

Jessup

Gust

2023

BMJ Health Care Inform

View full text Add to dashboard Cite

Background In the Australian public healthcare system, hospitals are funded based on the number of inpatient discharges and types of conditions treated (casemix). Demand for services is increasing faster than public funding and there is a need to identify and support patients that have high service usage. In 2016, the Victorian Department of Health and Human Services developed an algorithm to predict multiple unplanned admissions as part of a programme, Health Links Chronic Care (HLCC), that provided capitation funding instead of activity based funding to support patients with high admissions. Objectives The aim of this study was to determine whether an algorithm with higher performance than previously used algorithms could be developed to identify patients at high risk of three or more unplanned hospital admissions 12 months from discharge. Methods The HLCC and Hospital Unplanned Readmission Tool (HURT) models were evaluated using 34 801 unplanned inpatient episodes (27 216 patients) from 2017 to 2018 with an 8.3% prevalence of 3 or more unplanned admissions in the following year of discharge. Results HURT had a higher AUROC (84%, 95% CI 83.4% to 84.9% vs 71%, 95% CI 69.4% to 71.8%) than HLCC, that was statistically significant using Delong test at p<0.05. Discussion We found features that appear to be strong predictors of admission risk that have not been previously used in models, including socioeconomic status and social support. Conclusion The high AUROC, moderate sensitivity and high specificity for the HURT algorithm suggests it is a very good predictor of future multi-admission risk and that it can be used to provide targeted support for at-risk individual.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Novel machine learning model for predicting multiple unplanned hospitalisations

Jessup

Gust

2023

BMJ Health Care Inform

View full text Add to dashboard Cite

show abstract

“…We also repeated the analysis using SPARRAv4 [15], which is expected to be deployed in Scotland in 2024. SPARRAv4 was trained using more recent versions (2013-2018) of the same input data sources as SPARRAv3 and using more complex machine learning methods (e.g.…”

Section: Supplementary Notesmentioning

confidence: 99%

Differential behaviour of a risk score for emergency hospital admission by demographics in Scotland — a retrospective study

Thoma,

Rogers,

Ireland

et al. 2024

Preprint

Self Cite

View full text Add to dashboard Cite

The Scottish Patients at Risk of Re-Admission and Admission (SPARRA) score predicts individual risk of emergency hospital admission for approximately 80% of the Scottish population. It was developed using routinely collected electronic health records, and is used by primary care practitioners to inform anticipatory care, particularly for individuals with high healthcare needs. We comprehensively assess the SPARRA score across population subgroups defined by age, sex, ethnicity, socioeconomic deprivation, and geographic location. For these subgroups, we consider differences in overall performance, score distribution, and false positive and negative rates, using causal methods to identify effects mediated through age, sex, and deprivation. We show that the score is well-calibrated across subgroups, but that rates of false positives and negatives vary widely, mediated by a range of causes. Our work assists practitioners in the application and interpretation of the SPARRA score in population subgroups.

show abstract

“…The SPARRA series of risk scores predicts one-year risk of emergency hospital admission in the Scottish population on the basis of routinely collected electronic healthcare records [16]. Unlike QRISK, it is computed directly for (essentially) the entire population and deployed directly to general practitioners for the patients in their care.…”

Section: Sparra Score For Emergency Admissionsmentioning

confidence: 99%

Stacking interventions for equitable outcomes

Liley¹

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Predictive risk scores estimating probabilities for a binary outcome on the basis of observed covariates are frequently developed with the intent of avoiding that outcome by intervening on covariates in response to estimated risks. Since risk scores are usually developed in complex systems, interventions often take the form of expert actors responding to estimated risks as they best see fit. In this case, interventions may be complex and their effects difficult to observe or infer, meaning that explicit specification of interventions in response to risk scores is impractical. The capacity to design the aggregate modelintervention scheme in a way which optimises objectives is hence limited. We propose an algorithm by which a model-intervention scheme can be developed by 'stacking' possibly unknown intervention effects. By repeatedly observing and updating the intervention and model, this scheme leads to convergence or almost-convergence of eventual outcome risk to an equivocal value for any initial value of covariates, given reasonable assumptions. Roughly, our approach involves deploying a series of risk scores to expert actors, with instructions to act on them in succession. Our algorithm uses only observations of pre-intervention covariates and the eventual outcome as input. It is not necessary to know the action of the intervention, other than a general assumption that it is 'well-intentioned'. This algorithm can also be used to safely update risk scores in the presence of unknown interventions, an important contemporary problem in machine learning. We demonstrate convergence of expectation of outcome in a range of settings, and give sufficient conditions for convergence in distribution of covariate values. Finally, we demonstrate a potential practical implementation by simulation to optimise population-level outcome frequency.

show abstract

Development and assessment of a machine learning tool for predicting emergency admission in Scotland

Cited by 6 publications

References 39 publications

Novel machine learning model for predicting multiple unplanned hospitalisations

Novel machine learning model for predicting multiple unplanned hospitalisations

Differential behaviour of a risk score for emergency hospital admission by demographics in Scotland — a retrospective study

Stacking interventions for equitable outcomes

Contact Info

Product

Resources

About