A systematic review finds methodological improvements necessary for prognostic models in determining traumatic brain injury outcomes

Mushkudiani, Nino; Hukkelhoven, Chantal W.P.M.; Hernández, Adrian V.; Murray, Gordon D.; Choi, Sung C.; Maas, Andrew I. R.; Steyerberg, Ewout W.

doi:10.1016/j.jclinepi.2007.06.011

Cited by 151 publications

(142 citation statements)

References 57 publications

Supporting

Mentioning

125

Contrasting

Unclassified

Order By: Relevance

“…Common approaches include dichotomizing at the median value or choosing an optimal cutoff point based on minimizing a P value. Regardless of the approach used, the practice of artificially treating a continuous risk predictor as categorical should be avoided [23,24]; yet, this is often done in the development of risk prediction models [22,[24][25][26][27]. In this review, 86 % of studies were identified that all or some continuous risk predictors were dichotomized or categorized.…”

Section: Discussion Main Findingsmentioning

confidence: 99%

Developing Risk Prediction Models for Postoperative Pancreatic Fistula: a Systematic Review of Methodology and Reporting Quality

Zhang

Guo

et al. 2016

Indian J Surg

View full text Add to dashboard Cite

Postoperative pancreatic fistula is still a major complication after pancreatic surgery, despite improvements of surgical technique and perioperative management. We sought to systematically review and critically access the conduct and reporting of methods used to develop risk prediction models for predicting postoperative pancreatic fistula. We conducted a systematic search of PubMed and EMBASE databases to identify articles published before January 1, 2015, which described the development of models to predict the risk of postoperative pancreatic fistula. We extracted information of developing a prediction model including study design, sample size and number of events, definition of postoperative pancreatic fistula, risk predictor selection, missing data, modelbuilding strategies, and model performance. Seven studies of developing seven risk prediction models were included. In three studies (42 %), the number of events per variable was less than 10. The number of candidate risk predictors ranged from 9 to 32. Five studies (71 %) reported using univariate screening, which was not recommended in building a multivariate model, to reduce the number of risk predictors. Six risk prediction models (86 %) were developed by categorizing all continuous risk predictors. The treatment and handling of missing data were not mentioned in all studies. We found use of inappropriate methods that could endanger the development of model, including univariate pre-screening of variables, categorization of continuous risk predictors, and model validation. The use of inappropriate methods affects the reliability and the accuracy of the probability estimates of predicting postoperative pancreatic fistula.

show abstract

Section: Discussion Main Findingsmentioning

confidence: 99%

Developing Risk Prediction Models for Postoperative Pancreatic Fistula: a Systematic Review of Methodology and Reporting Quality

Zhang

Guo

et al. 2016

Indian J Surg

View full text Add to dashboard Cite

show abstract

“…1) The initial examination included a set of 15 potential predictors ( Table 1) that were collected in the first 6 hours after a TBI; these predictors are consistently referenced in the literature. 5,16,33 The relationship between a predictor and early death was estimated using a crude odds ratio and R 2 . 34 2) The full model with 15 predictors was developed using penalized maximum likelihood estimation to directly correct the model for overoptimism.…”

Section: Model Development and Statistical Analysesmentioning

confidence: 99%

“…25 A common problem in using prognostic systems is that the systems are not generalizable. 40 The systematic reviews of prognostic models for TBIs that have been published in the past 20 years have generated criticism regarding their clinical or methodological validities; 15,33 only a few studies have been subjected to extensive internal and external validations. 2,10,16,36,37,39,42 Few studies have analyzed the characteristics of patients who succumbed to early death after a TBI.…”

mentioning

confidence: 99%

Validation of a prognostic score for early mortality in severe head injury cases

Gómez¹,

De-la-Cruz

Lora

et al. 2014

JNS

View full text Add to dashboard Cite

Object Traumatic brain injury (TBI) represents a large health and economic burden. Because of the inability of previous randomized controlled trials (RCTs) on TBI to demonstrate the expected benefit of reducing unfavorable outcomes, the IMPACT (International Mission on Prognosis and Analysis of Clinical Trials in TBI) and CRASH (Corticosteroid Randomisation After Significant Head Injury) studies provided new methods for performing prognostic studies of TBI. This study aimed to develop and externally validate a prognostic model for early death (within 48 hours). The secondary aim was to identify patients who were more likely to succumb to an early death to limit their inclusion in RCTs and to improve the efficiency of RCTs. Methods The derivation cohort was recruited at 1 center, Hospital 12 de Octubre, Madrid (1990–2003, 925 patients). The validation cohort was recruited in 2004–2006 from 7 study centers (374 patients). The eligible patients had suffered closed severe TBIs. The study outcome was early death (within 48 hours post-TBI). The predictors were selected using logistic regression modeling with bootstrapping techniques, and a penalized reduction was used. A risk score was developed based on the regression coefficients of the variables included in the final model. Results In the validation set, the final model showed a predictive ability of 50% (Nagelkerke R2), with an area under the receiver operating characteristic curve of 89% and an acceptable calibration (goodness-of-fit test, p = 0.32). The final model included 7 variables, and it was used to develop a risk score with a range from 0 to 20 points. Age provided 0, 1, 2, or 3 points depending on the age group; motor score provided 0 points, 2 (untestable), or 3 (no response); pupillary reactivity, 0, 2 (1 pupil reacted), or 6 (no pupil reacted); shock, 0 (no) or 2 (yes); subarachnoid hemorrhage, 0 or 1 (severe deposit); cisternal status, 0 or 3 (compressed/absent); and epidural hematoma, 0 (yes) or 2 (no). Based on the risk of early death estimated with the model, 4 risk of early death groups were established: low risk, sum score 0–3 (< 1% predicted mortality); moderate risk, sum score 4–8 (predicted mortality between 1% and 10%); high risk, sum score 9–12 (probability of early death between 10% and 50%); and very high risk, sum score 13–20 (early mortality probability > 50%). This score could be used for selecting patients for clinical studies. For example, if patients with very high risk scores were excluded from our study sample, the patients included (eligibility score < 13) would represent 80% of the original sample and only 23% of the patients who died early. Conclusions The combination of Glasgow Coma Scale score, CT scanning results, and secondary insult data into a prognostic score improved the prediction of early death and the classification of TBI patients.

show abstract

“…Several prognostic models to predict outcome in TBI patients exist, but few meet the methodological requirements for valid prognostic model development, including a sufficiently large development sample and internal or external validation. 4,5 A set of prognostic models meeting these requirements is the International Mission on Prognosis and Analysis of Clinical Trials in TBI (IMPACT) models for the prediction of 6-month mortality and unfavorable outcome. These models were developed in almost 10,000 patients and include demographic variables (age), parameters of brain injury severity, computed tomographic (CT) findings and laboratory values as predictors.…”

mentioning

confidence: 99%

Prognosis in moderate and severe traumatic brain injury

Lingsma

Andriessen

Haitsema

et al. 2013

Journal of Trauma and Acute Care Surgery

Self Cite

102

View full text Add to dashboard Cite

BACKGROUND:Several prognostic models to predict outcome in traumatic brain injury (TBI) have been developed, but few are externally validated. We aimed to validate the International Mission on Prognosis and Analysis of Clinical Trials in TBI (IMPACT) prognostic models in a recent unselected patient cohort and to assess the additional prognostic value of extracranial injury. METHODS:The Prospective Observational COhort Neurotrauma (POCON) registry contains 508 patients with moderate or severe TBI, who were admitted in 2008 and 2009 to five trauma centers in the Netherlands. We predicted the probability of mortality and unfavorable outcome at 6 months after injury with the IMPACT prognostic models. We studied discrimination (area under the curve [AUC]) and calibration. We added the extracranial component of the Injury Severity Score (ISS) to the models and calculated the increase in AUC. RESULTS:The IMPACT models had an adequate discrimination in the POCON registry, with AUCs in the external validation between 0.85 and 0.90 for mortality and between 0.82 and 0.87 for unfavorable outcome. Observed outcomes agreed well with predicted outcomes. Adding extracranial injury slightly improved predictions in the overall population (unfavorable outcome: AUC increase of 0.002, p = 0.02; mortality: AUC increase of 0.000, p = 0.37) but more clearly in patients with moderate TBI (unfavorable outcome: AUC increase of 0.008, p G 0.01, mortality: AUC increase of 0.012, p = 0.02) and patients with minor computed tomographic result abnormalities (unfavorable outcome: AUC increase of 0.013, p G 0.01; mortality: AUC increase of 0.001, p = 0.08). CONCLUSION:The IMPACT models performed well in a recent series of TBI patients. We found some additional impact of extracranial injury on outcome, specifically in patients with less severe TBI or minor computed tomographic result abnormalities. (J Trauma Acute Care Surg. 2013;74: 639Y646.

show abstract

A systematic review finds methodological improvements necessary for prognostic models in determining traumatic brain injury outcomes

Cited by 151 publications

References 57 publications

Developing Risk Prediction Models for Postoperative Pancreatic Fistula: a Systematic Review of Methodology and Reporting Quality

Developing Risk Prediction Models for Postoperative Pancreatic Fistula: a Systematic Review of Methodology and Reporting Quality

Validation of a prognostic score for early mortality in severe head injury cases

Prognosis in moderate and severe traumatic brain injury

Contact Info

Product

Resources

About