Improving Clinical Utility of Real-World Prediction Models: Updating Through Recalibration

Abstract

Prediction models can aid clinicians in identifying at-risk athletes. However, sport and clinical practice patterns continue to change, causing predictive drift and potential suboptimal prediction model performance. Thus, there is a need to temporally recalibrate previously developed baseball arm injury models. The purpose of this study was to perform temporal recalibration on a previously developed injury prediction model and assess model performance in professional baseball pitchers. An arm injury prediction model was developed on data from a prospective cohort from 2009 to 2019 on minor league pitchers. Data for the 2015-2019 seasons were used for temporal recalibration and model performance assessment. Temporal recalibration constituted intercept-only and full model redevelopment. Model performance was investigated by assessing Nagelkerke's R-square, calibration in the large, calibration, and discrimination. Decision curves compared the original model, temporal recalibrated model, and current best evidencebased practice. One hundred seventy-eight pitchers participated in the 2015-2019 seasons with 1.63 arm injuries per 1,000 athlete exposures. The temporal recalibrated intercept model demonstrated the best discrimination (0.81 [95% confidence interval [CI]: 0.73, 0.88]) and R-square (0.32) compared with original model (0.74 [95% CI: 0.69, 0.80]; R-square: 0.32) and the redeveloped model (0.80 [95% CI: 0.73, 0.87]; R-square: 0.30). The temporal recalibrated intercept model demonstrated an improved net benefit of 0.34 compared with current best evidence-based practice. The temporal recalibrated intercept model demonstrated the best model performance and clinical utility. Updating prediction models can account for changes in sport training over time and improve professional baseball arm injury outcomes.