Clinical Prediction Tools in Trauma: Where Do We Go From Here?

Abstract

Expeditious hemorrhage control is a fundamental tenet of trauma surgery. Every minute delayed before an emergency laparotomy for hemorrhage control has been associated with an increased probability of mortality. 1 Gutierrez et al 2 developed and validated a much needed tool to identify patients at risk of requiring early laparotomy within 2 hours of hospital arrival after injury. The Prehopsital Preparation for Surgery (PREPS) Score was derived using encounters of adults aged 18 years or older in the 2017 Trauma Quality Improvement Program database. The PREPS Score (0-20) comprised 7 binary variables, with factors more strongly associated with early laparotomy contributing more points and input variables that could be used by emergency medical service (EMS) technicians in the field.Building this type of clinical tool requires thoughtful study design. First, the tool should have the potential to alter clinical decision-making. The PREP Score certainly meets this criterion; early identification of patients requiring emergency laparotomy may help hospitals mobilize resources, ensure patients are triaged to appropriate centers, and alert clinicians to patients with a higher likelihood of deterioration. Second, variable selection must follow an a priori determined and statistically sound approach. Including too many variables in a model risks overfitting and lacks practicality. In addition to the variable selection strategy used by Gutierrez et al, 2 other strategies can be used, ranging from removing variables with near-zero variance to regularization and other machine learning algorithms. [3][4][5] Third, model validity must be assessed using appropriate performance metrics. Various performance metrics exist, including the C-statistic, which the authors used. The C-statistic is equivalent to area under the receiver operating characteristic curve (AUROC) for binary outcomes, a frequently reported yet commonly misunderstood value. Derived from the area under the curve plotting sensitivity (true positive rate) vs 1 − specificity (false positive rate), AUROC denotes the probability that a randomly selected point in the positive class will be scored higher than a randomly selected point in the negative class. While the concept that a higher C-statistic or AUROC close to 1 indicates good performance, applying the value itself for clinical decision-making is challenging.Alternative performance metrics rather than AUROC or C-statistic may be more interpretable.For example, if the cost of a false negative is high (eg, most screening tools), sensitivity or recall would be an appropriate performance metric to evaluate, while precision would be the more informative metric if the cost of a false positive is high (eg, if positive result necessitates invasive or high-risk intervention). Class imbalance (the ratio of positives to negatives in the study population) is an important consideration when choosing the performance metric. For example, in scenarios with few positives, accuracy would be a misleading metric. If a tool always ...

DeepBackRib: Deep learning to understand factors associated with readmissions after rib fractures

“…Sophisticated machine-learning algorithms and high area under the receiver operating characteristic curves (an inappropriate performance metric for many clinical prediction tasks, which use imbalanced datasets 15 ) are insufficient for yielding useful clinical prediction tools. The MARA framework should inform the algorithm/analysis choice, and prediction tool development/validation should follow the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis guideline.…”

“…We report recall, precision, area under the receiver operator characteristic curve, and area under the precision recall curve (AUPRC). We report AUPRC since area under the receiver operator characteristic curve alone can be misleading when using imbalanced datasets 15 …”

“…We report AUPRC since area under the receiver operator characteristic curve alone can be misleading when using imbalanced datasets. 15…”

“…Similar to odds ratios within multivariable logistic regression models, SHAP analysis facilitates understanding which factors may be associated with an outcome of interest. Rather than a model that only outputs binary predictions, deep learning should be considered as another modality to better understand or find new associations between predictors and outcomes.Sophisticated machine-learning algorithms and high area under the receiver operating characteristic curves (an inappropriate performance metric for many clinical prediction tasks, which use imbalanced datasets15 ) are insufficient for yielding useful clinical prediction tools. The MARA framework should inform the algorithm/analysis choice, and prediction tool development/validation should follow the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis guideline.…”

Longitudinal analysis of neutrophil-to-lymphocyte ratio over time after major trauma

“…It is possible that the addition of NLR to other tools may provide more robust outcome predictions that are dynamic and responsive to management. 15 As full blood examinations are routinely performed regularly on critically injured patients, the NLR provides an objective measure of injury severity that can be easily determined. Addition of the NLR to more complex prediction tools may result in robust outcome predictions after trauma and initial resuscitation.…”