Assessment of Time-Series Machine Learning Methods for Forecasting Hospital Discharge Volume

McCoy, Thomas H.; Pellegrini, Amelia M.; Perlis, Roy H.

doi:10.1001/jamanetworkopen.2018.4087

Cited by 42 publications

(32 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Furthermore, some models that have predicted daily discharges have done so across all patients in the hospital, which is potentially useful at the hospital level but not actionable for clinical teams caring for the individual patient. 18,19 Limitations Our study has some limitations. The model was trained and validated on patients within a single institution; thus, its generalizability to other hospitals is uncertain.…”

Section: Discussionmentioning

confidence: 94%

Development and Validation of a Machine Learning Model to Aid Discharge Processes for Inpatient Surgical Care

et al. 2019

View full text Add to dashboard Cite

IMPORTANCE Inpatient overcrowding is associated with delays in care, including the deferral of surgical care until beds are available to accommodate postoperative patients. Timely patient discharge is critical to address inpatient overcrowding and requires coordination among surgeons, nurses, case managers, and others. This is difficult to achieve without early identification and systemwide transparency of discharge candidates and their respective barriers to discharge. OBJECTIVE To validate the performance of a clinically interpretable feedforward neural network model that could improve the discharge process by predicting which patients would be discharged within 24 hours and their clinical and nonclinical barriers. DESIGN, SETTING, AND PARTICIPANTS This prognostic study included adult patients discharged from inpatient surgical care from May 1, 2016, to August 31, 2017, at a quaternary care teaching hospital. Model performance was assessed with standard cross-validation techniques. The model's performance was compared with a baseline model using historical procedure median length of stay to predict discharges. In prospective cohort analysis, the feedforward neural network model was used to make predictions on general surgical care floors with 63 beds. If patients were not discharged when predicted, the causes of delay were recorded. MAIN OUTCOMES AND MEASURES The primary outcome was the out-of-sample area under the receiver operating characteristic curve of the model. Secondary outcomes included the causes of discharge delay and the number of avoidable bed-days. RESULTS The model was trained on 15 201 patients (median [interquartile range] age, 60 [46-70] years; 7623 [50.1%] men) discharged from inpatient surgical care. The estimated out-of-sample area under the receiver operating characteristic curve of the model was 0.840 (SD, 0.008; 95% CI, 0.839-0.844). Compared with the baseline model, the neural network model had higher sensitivity (52.5% vs 56.6%) and specificity (51.7% vs 82.6%). The neural network model identified 65 barriers to discharge. In the prospective study of 605 patients, causes of delays included clinical barriers (41 patients [30.1%]), variation in clinical practice (30 patients [22.1%]), and nonclinical reasons (65 patients [47.8%]). Summing patients who were not discharged owing to variation in clinical practice and nonclinical reasons, 128 bed-days, or 1.2 beds per day, were classified as avoidable. CONCLUSIONS AND RELEVANCE This cohort study found that a neural network model could predict daily inpatient surgical care discharges and their barriers. The model identified systemic causes of discharge delays. Such models should be studied for their ability to increase the timeliness of discharges.

show abstract

Section: Discussionmentioning

confidence: 94%

Development and Validation of a Machine Learning Model to Aid Discharge Processes for Inpatient Surgical Care

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Scope Predicted outcome Methodology Tabak et al (2014) All inpatients Mortality Linear regression Van Walraven and Forster (2017) All inpatients Discharge volume Survival analysis McCoy et al (2018) Hospital level Discharge volume Time series Rajkomar et al (2018) All inpatients Mortality, overall LOS > 7 days Deep learning Safavi et al (2019) Surgical inpatients Remaining LOS < 1 day Deep learning most severe patients. Patients who cannot be admitted to an ICU due to congestion have to be admitted to a general care bed, leading to increased length of stay and readmission risk (Kim et al 2016).…”

Section: Referencementioning

confidence: 99%

“…Recently, Van Walraven and Forster (2017) use non-parametric models to predict the daily number of hospital discharges with accuracy roughly comparable to ours (median relative error: 1.4%; IQR −5.5% to 7.1%). McCoy et al (2018) forecasted discharge volume in two hospitals using time-series algorithm with a mean absolute error of 11.5 and 11.7 beds respectively ( R 2 = 0.843 and 0.726 respectively).…”

Section: Predictive Accuracy On Retrospective Datamentioning

confidence: 99%

See 1 more Smart Citation

Predicting inpatient flow at a major hospital using interpretable analytics

Bertsimas

Pauphilet

Stevens

et al. 2020

Preprint

View full text Add to dashboard Cite

Problem definition: Turn raw data from Electronic Health Records into accurate predictions on patient flows and inform daily decision-making at a major hospital. Practical Relevance: In a hospital environment under increasing financial and operational stress, forecasts on patient demand patterns could help match capacity and demand and improve hospital operations. Methodology: We use data from 63,432 admissions at a large academic hospital (50.0% female, median age 64 years old, median length-of-stay 3.12 days). We construct an expertise-driven patient representation on top of their EHR data and apply a broad class of machine learning methods to predict several aspects of patient flows. Results: With a unique patient representation, we estimate short-term discharges, identify long-stay patients, predict discharge destination and anticipate flows in and out of intensive care units with accuracy in the 80%+ range. More importantly, we implement this machine learning pipeline into the EHR system of the hospital and construct prediction-informed dashboards to support daily bed placement decisions. Managerial Implications: Our study demonstrates that interpretable machine learning techniques combined with EHR data can be used to provide visibility on patient flows. Our approach provides an alternative to deep learning techniques which is equally accurate, interpretable, frugal in data and computational power, and production-ready.

show abstract

“…16 17 This has motivated operationsfocused researchers to develop automated predictions of individual discharges or discharge volume using data available in the electronic health record (EHR). [17][18][19][20] These retrospective analyses have demonstrated that EHR-data-driven models can perform as well-and sometimes better-than clinician-driven predictions. 17 However, to our knowledge, there have been limited published studies demonstrating translation of an automated discharge prediction model into clinical practice with analysis of impact on operational outcomes.…”

Section: Introductionmentioning

confidence: 99%

Machine-learning-based hospital discharge predictions can support multidisciplinary rounds and decrease hospital length-of-stay

et al. 2020

View full text Add to dashboard Cite

BackgroundPatient flow directly affects quality of care, access and financial performance for hospitals. Multidisciplinary discharge-focused rounds have proven to minimise avoidable delays experienced by patients near discharge. The study objective was to support discharge-focused rounds by implementing a machine-learning-based discharge prediction model using real-time electronic health record (EHR) data. We aimed to evaluate model predictive performance and impact on hospital length-of-stay.MethodsDischarge prediction models were developed from hospitalised patients on four inpatient units between April 2016 and September 2018. Unit-specific models were implemented to make individual patient predictions viewable with the EHR patient track board. Predictive performance was measured prospectively for 12 470 patients (120 780 patient-predictions) across all units. A pre/poststudy design applying interrupted time series methods was used to assess the impact of the discharge prediction model on hospital length-of-stay.ResultsProspective discharge prediction performance ranged in area under the receiver operating characteristic curve from 0.70 to 0.80 for same-day and next-day predictions; sensitivity was between 0.63 and 0.83 and specificity between 0.48 and 0.80. Elapsed length-of-stay, counts of labs and medications, mobility assessments and measures of acute kidney injury were model features providing the most predictive value. Implementing the discharge predictions resulted in a reduction in hospital length-of-stay of over 12 hours on a medicine unit (p<0.001) and telemetry unit (p=0.002), while no changes were observed for the surgery unit (p=0.190) and second medicine unit (p<0.555).ConclusionsIncorporating automated patient discharge predictions into multidisciplinary rounds can support decreases in hospital length-of-stay. Variation in execution and impact across inpatient units existed.

show abstract

Assessment of Time-Series Machine Learning Methods for Forecasting Hospital Discharge Volume

Cited by 42 publications

References 30 publications

Development and Validation of a Machine Learning Model to Aid Discharge Processes for Inpatient Surgical Care

Development and Validation of a Machine Learning Model to Aid Discharge Processes for Inpatient Surgical Care

Predicting inpatient flow at a major hospital using interpretable analytics

Machine-learning-based hospital discharge predictions can support multidisciplinary rounds and decrease hospital length-of-stay

Contact Info

Product

Resources

About