A critical appraisal of the clinical applicability and risk of bias of the predictive models for mortality and recurrence in patients with oropharyngeal cancer: Systematic review

Abstract

The use of predictive models is becoming widespread. However, these models should be developed appropriately (CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modeling Studies [CHARMS] and Prediction model Risk Of Bias ASsessment Tool [PROBAST] statements). Concerning mortality/recurrence in oropharyngeal cancer, we are not aware of any systematic reviews of the predictive models. We carried out a systematic review of the MEDLINE/EMBASE databases of those predictive models. In these models, we analyzed the 11 domains of the CHARMS statement and the risk of bias and applicability, using the PROBAST tool. Six papers were finally included in the systematic review and all of them presented high risk of bias and several limitations in the statistical analysis. The applicability was satisfactory in five out of six studies. None of the models could be considered ready for use in clinical practice.

A critical appraisal of the prognostic predictive models for patients with sepsis: Which model can be applied in clinical practice?

“…This was grouped into 11 domains: (1) Source of data (study design); (2) Participants (description, dates and treatments); (3) Outcome to be predicted (definition of the outcome to be predicted and period of onset); (4) candidate predictors (number and type, form and time of measurement, and handling of continuous predictors); (5) Sample size (number of participants, number of events and the event‐per‐variable ratio); (6) Missing data (number of participants with missing data overall and for each predictor, and handling of missing data); (7) Model development (type and methods used for the selection of predictors); (8) Model performance (calibration and discrimination); (9) Model evaluation (method used); (10) Results (form of presentation of the final model [nomogram, points system, mathematical formula, etc], weight of the different variables, and comparison of development and external validation samples when appropriate) and (11) Interpretation and discussion (comparison with similar studies, generalisation of the model, strengths and limitations) 9 . All this information is presented in tabular form to make it easier to see all the characteristics of the models 13,14 …”

Prediction Models for Prognosis of Cervical Cancer: Systematic Review and Critical Appraisal

“…Our systematic review showed several methodological pitfalls in the development or validation of the models, especially in the analysis domain which was the main cause of high ROB. Similar to reviews assessing the quality of prognostic models for other diseases including oropharyngeal cancer, chronic lymphocytic leukemia, and chronic obstructive pulmonary disease (19,20,39,40), inadequate sample size, improper handling of missing data, and incomplete evaluation of model performance were the main problems in analysis domain and needed to be emphasized. Besides this, several other methodological details should also be noted.…”

“…As for cervical cancer patients, due to the limited predictive value for the classification of the International Federation of Gynecology and Obstetrics (FIGO) alone, a couple of prognostic models have been proposed to predict and guide treatments based on different tumor and demographic characteristics (13,18). However, the uneven quality and the diversity of the clinical settings, outcomes, and predictors may limit the practicality of models, and systematic reviews on prognostic models of other diseases also suggested that the methodological features of existing studies varied (19)(20)(21). No comprehensive evaluation of prognostic models for cervical cancer has been done.…”

A general presentation on how to carry out aCHARMSanalysis for prognostic multivariate models

“…The comparison with the existing literature should be performed based on other reviews of prediction models with similar diseases and/or populations, in order to compare the adherence to the CHARMS recommendations. Finally, for the implications for research and clinical practice we must be cautious, because there are 11 domains in the CHARMS checklist, and it is difficult to have a model fulfilling all the requirements 13,14 . Obviously, if we use “the worst score counts” algorithm, no model could be applied in the real world.…”

“…Although the CHARMS checklist was published in 2014 and several systematic reviews of predictive models have been conducted to date, 3,4,10‐14 as far as we know, the studies usually only undertake an overall descriptive analysis of the results of each article included in the review, 3,4,10‐12 rather than an extensive analysis of each predictive model, as our working group does 13,14 . This is a key issue, since when assessing a particular prediction model, we need to know all the details of its development, validation and how to apply it to new patients clearly and simply.…”