A critical appraisal of the prognostic predictive models for patients with sepsis: Which model can be applied in clinical practice?

Abstract

Background: Sepsis is associated with high mortality and predictive models can help in clinical decision-making. The objective of this study was to carry out a systematic review of these models. Methods:In 2019, we conducted a systematic review in MEDLINE and EMBASE (CDR42018111121:PROSPERO) of articles that developed predictive models for mortality in septic patients (inclusion criteria). We followed the CHARMS recommendations (Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies), extracting the information from its 11 domains (Source of data, Participants, etc). We determined the risk of bias and applicability (participants, outcome, predictors and analysis) through PROBAST (Prediction model Risk Of Bias ASsessment Tool).Results: A total of 14 studies were included. In the CHARMS extraction, the models found showed great variability in its 11 domains. Regarding the PROBAST checklist, only one article had an unclear risk of bias as it did not indicate how missing data were handled while the others all had a high risk of bias. This was mainly due to the statistical analysis (inadequate sample size, handling of continuous predictors, missing data and selection of predictors), since 13 studies had a high risk of bias. Applicability was satisfactory in six articles. Most of the models integrate predictors from routine clinical practice. Discrimination and calibration were assessed for almost all the models, with the area under the ROC curve ranging from 0.59 to 0.955 and no lack of calibration. Only three models were externally validated and their maximum discrimination values in the derivation were from 0.712 and 0.84. One of them (Osborn) had undergone multiple validation studies.Discussion: Despite most of the studies showing a high risk of bias, we very cautiously recommend applying the Osborn model, as this has been externally validated various times.

Mortality prediction models after radical cystectomy for bladder tumour: A systematic review and critical appraisal

“…Our systematic review shows that existing models in the literature have a high risk of bias and low clinical applicability. These findings are consistent with others, a previous systematic review by Beneyto et al, 23 which identified and summarized, through the use of the PROBAST criteria the predictive models for predicting mortality in sepsis. The included studies showed a high risk of bias in the Participants, Predictors, Outcome and Analysis domains, with the risk of bias in the latter domain being high in 80%–100% of the studies.…”

“…20,21 Both tools have been widely used in several diseases, showing limitations and difficulty of models to be applied in clinical practice when they have biases. [22][23][24] As far as we know, no systematic review of prediction models of UBC mortality has been carried out with the application of CHARMS and PROBAST. [19][20][21] Hence, a summary of the existing models is lacking, including the description of the risk of bias in each model, to allow clinicians to better stratify the mortality risk of these patients.…”

“…Five years later, Prediction Model Risk Of Bias Assessment Tool (PROBAST) was published to assess the clinical applicability and risk of bias of prediction models, based on the results obtained from a CHARMS review 20,21 . Both tools have been widely used in several diseases, showing limitations and difficulty of models to be applied in clinical practice when they have biases 22–24 …”

A Novel Decision Tree Model for Predicting the Cancer-Specific Survival of Patients with Bladder Cancer Treated with Radical Cystectomy

“…In this systematic review, the authors provide a synthesis of the 19 prediction models identified. Among them, 52.6% showed low applicability and all of them had a high risk of bias; these findings agree with previous systematic reviews carried out with a similar methodology [6,14].…”

An Analysis of Predictive Factors for Severe Neonatal Infection and the Construction of a Prediction Model

“…2 Thus, identifying clinical indicators that can predict severe neonatal infection early and effectively and establishing predictive models based on these indicators is of great clinical significance for the early diagnosis, control and improved prognosis of neonatal infection. [3][4][5] White blood cell count (WBC), CRP and PCT levels are currently used to diagnose and monitor neonatal infection, 6 white blood cells are essential components of the human immune system, responsible for identifying and eliminating pathogens such as bacteria, viruses, and parasites, when an infection occurs, white blood cells increase to counter the infection, platelets play a key role in coagulation and hemostasis while also participating in inflammatory responses. During an infection, the inflammatory response can lead to platelet activation and consumption, potentially resulting in a decreased platelet count, additionally, some infections (particularly bacterial infections) can cause a reduction in platelets as toxins or immune mediators produced by the pathogens may directly damage platelets or interfere with their production, but the sensitivity and specificity of any single peripheral blood index need to be studied further.…”