Statistical Development and Validation of Clinical Prediction Models

Abstract

Summary Clinical prediction models in anesthesia and surgery research have many clinical applications including preoperative risk stratification with implications for clinical utility in decision-making, resource utilization, and costs. It is imperative that predictive algorithms and multivariable models are validated in a suitable and comprehensive way in order to establish the robustness of the model in terms of accuracy, predictive ability, reliability, and generalizability. The purpose of this article is to educate anesthesia researchers at an introductory level on important statistical concepts involved with development and validation of multivariable prediction models for a binary outcome. Methods covered include assessments of discrimination and calibration through internal and external validation. An anesthesia research publication is examined to illustrate the process and presentation of multivariable prediction model development and validation for a binary outcome. Properly assessing the statistical and clinical validity of a multivariable prediction model is essential for reassuring the generalizability and reproducibility of the published tool.

The Promise and Challenges of Predictive Analytics in Perioperative Care

“…Based on analysis of simulated data, 11 Enevoldsen and Vistisen show that such selection bias substantially overestimates the performance of current MAP in predicting future hypotension (AUC increased from marginally useful 0.75 to highly useful 0.93). 6 In these same simulated data, selection bias increased the specificity of current MAP less than 75 mmHg in predicting future hypotension from about 70% to essentially perfect 100%, while sensitivity remained unchanged at about 70%.…”

Establishment and Validation of a Predictive Nomogram for Hallux Valgus with Pain Under the Second Metatarsal

“…As a quantitative tool for risk and benefit assessment, it can provide intuitive and rational information for decision making by doctors, patients and healthcare personnel. 38 In the prevention and treatment of hallux valgus foot combined with pain under the second metatarsal head, the forefoot metatarsal pad can be used to alleviate the change in the arch of the hallux valgus foot through changes in imaging and plantar pressure in the early stage to indirectly reduce the load of the second metatarsal head and delay the aggravation of the HVA through the use of a hallux valgus correction belt or a split toe pad. Of course, it is very important to avoid soft tissue factors, such as the strain on internal and external muscles and ligaments, through internal and external muscle training.…”

Protocol for the derivation and external validation of a 30-day postoperative pulmonary complications (PPCs) risk prediction model for elderly patients undergoing thoracic surgery: a cohort study in southern China

“…We assumed that this sample size would be adequately powered to derive our development model. Based on previous literature and a retrospective study in our centre, an anticipated incidence of 30-day PPCs was approximately 40% in a mixed cohort of elderly patients undergoing thoracic procedures 4 8–10. Assumption of a required number of 10 events per variable to be estimated by a logistic regression model, the size of development cohort will be determined to be at least 1000 patients.…”

“…Predictive models have been applied to identify patients at risk for postoperative complications by healthcare providers and policymakers, so as to formulate individualised preventive algorithms and allocate healthcare resource efficiently. 10 Moreover, the selected methodology must be stringent and comply with the guidelines of good clinical practice to make the model accurate and generalisable. A systematic review identified 21 prediction models for PPC, however, none of them completely abided by the recommended steps for the model derivation and validation.…”