Machine Learning for the Prediction of Procedural Case Durations Developed Using a Large Multicenter Database: Algorithm Development and Validation Study

Abstract

Background Accurate projections of procedural case durations are complex but critical to the planning of perioperative staffing, operating room resources, and patient communication. Nonlinear prediction models using machine learning methods may provide opportunities for hospitals to improve upon current estimates of procedure duration. Objective The aim of this study was to determine whether a machine learning algorithm scalable across multiple centers could make estimations of case duration within a tolerance limit because there are substantial resources required for operating room functioning that relate to case duration. Methods Deep learning, gradient boosting, and ensemble machine learning models were generated using perioperative data available at 3 distinct time points: the time of scheduling, the time of patient arrival to the operating or procedure room (primary model), and the time of surgical incision or procedure start. The primary outcome was procedure duration, defined by the time between the arrival and the departure of the patient from the procedure room. Model performance was assessed by mean absolute error (MAE), the proportion of predictions falling within 20% of the actual duration, and other standard metrics. Performance was compared with a baseline method of historical means within a linear regression model. Model features driving predictions were assessed using Shapley additive explanations values and permutation feature importance. Results A total of 1,177,893 procedures from 13 academic and private hospitals between 2016 and 2019 were used. Across all procedures, the median procedure duration was 94 (IQR 50-167) minutes. In estimating the procedure duration, the gradient boosting machine was the best-performing model, demonstrating an MAE of 34 (SD 47) minutes, with 46% of the predictions falling within 20% of the actual duration in the test data set. This represented a statistically and clinically significant improvement in predictions compared with a baseline linear regression model (MAE 43 min; P<.001; 39% of the predictions falling within 20% of the actual duration). The most important features in model training were historical procedure duration by surgeon, the word “free” within the procedure text, and the time of day. Conclusions Nonlinear models using machine learning techniques may be used to generate high-performing, automatable, explainable, and scalable prediction models for procedure duration.

Consolidated Reporting Guidelines for Prognostic and Diagnostic Machine Learning Models (CREMLS)

“…Describes how the prediction model fits in the clinical practice of scheduling operating theater procedures [5] The intended use of the ML model 1.8…”

“…Defined in Figure 1 in the paper by Kendale et al [5] Inclusion or exclusion criteria for the patient cohort 2.1 Describes sources and methods of data collection, what type of data were used, and potential implied bias in interpretation [23] Methods of data collection 2.2 Discusses potential bias in data collection and outcome definition [23] Bias introduced due to the method of data collection used 2.3…”

“…Missingness and outlier detection were discussed [5] Known quality issues with the data 2.6 Brief section dedicated to power analysis [5] Sample size calculation 2.7…”

“…Many papers reporting the use of ML models in medicine have used a large clinical data set to make diagnostic or prognostic predictions [3][4][5][6]. However, the use of data from electronic health records and other resources is often not without pitfalls as these data are typically collected and optimized for other purposes (eg, medical billing) [7].…”

Prediction of remaining surgery duration in laparoscopic videos based on visual saliency and the transformer network

“…of open surgery 7,8,9 and minimally invasive operations 10,11 . The employed data involve preoperative features such as patient and operation characteristics, time parameters, the surgeon's experience, and the equipment status.…”