ObjectiveTo compare performance of risk prediction models for forecasting postoperative sepsis and acute kidney injury.DesignRetrospective single center cohort study of adult surgical patients admitted between 2000 and 2010.Patients50,318 adult patients undergoing major surgery.MeasurementsWe evaluated the performance of logistic regression, generalized additive models, naïve Bayes and support vector machines for forecasting postoperative sepsis and acute kidney injury. We assessed the impact of feature reduction techniques on predictive performance. Model performance was determined using the area under the receiver operating characteristic curve, accuracy, and positive predicted value. The results were reported based on a 70/30 cross validation procedure where the data were randomly split into 70% used for training the model and the 30% for validation.Main ResultsThe areas under the receiver operating characteristic curve for different models ranged between 0.797 and 0.858 for acute kidney injury and between 0.757 and 0.909 for severe sepsis. Logistic regression, generalized additive model, and support vector machines had better performance compared to Naïve Bayes model. Generalized additive models additionally accounted for non-linearity of continuous clinical variables as depicted in their risk patterns plots. Reducing the input feature space with LASSO had minimal effect on prediction performance, while feature extraction using principal component analysis improved performance of the models.ConclusionsGeneralized additive models and support vector machines had good performance as risk prediction model for postoperative sepsis and AKI. Feature extraction using principal component analysis improved the predictive performance of all models.

When designing a clinical trial, borrowing historical control information can provide a more efficient approach by reducing the necessary control arm sample size while still yielding increased power. Several Bayesian methods for incorporating historical information via a prior distribution have been proposed, for example, (modified) power prior, (robust) meta‐analytic predictive prior. When utilizing historical control borrowing, the prior parameter(s) must be specified to determine the magnitude of borrowing before the current data are observed. Thus, a flexible prior is needed in case of heterogeneity between historic trials or prior data conflict with the current trial. To incorporate the ability to selectively borrow historic information, we propose a Bayesian semiparametric meta‐analytic‐predictive prior. Using a Dirichlet process mixture prior allows for relaxation of parametric assumptions, and lets the model adaptively learn the relationship between the historic and current control data. Additionally, we generalize a method for estimating the prior effective sample size (ESS) for the proposed prior. This gives an intuitive quantification of the amount of information borrowed from historical trials, and aids in tuning the prior to the specific task at hand. We illustrate the effectiveness of the proposed methodology by comparing performance between existing methods in an extensive simulation study and a phase II proof‐of‐concept trial in ankylosing spondylitis. In summary, our proposed robustification of the meta‐analytic‐predictive prior alleviates the need for prespecifying the amount of borrowing, providing a more flexible and robust method to integrate historical data from multiple study sources in the design and analysis of clinical trials.

As the availability of real‐world data sources (eg, EHRs, claims data, registries) and historical data has rapidly surged in recent years, there is an increasing interest and need from investigators and health authorities to leverage all available information to reduce patient burden and accelerate both drug development and regulatory decision making. Bayesian meta‐analytic approaches are a popular historical borrowing method that has been developed to leverage such data using robust hierarchical models. The model structure accounts for various degrees of between‐trial heterogeneity, resulting in adaptively discounting the external information in the case of data conflict. In this article, we propose to integrate the propensity score method and Bayesian meta‐analytic‐predictive (MAP) prior to leverage external real‐world and historical data. The propensity score methodology is applied to select a subset of patients from external data that are similar to those in the current study with regards to key baseline covariates and to stratify the selected patients together with those in the current study into more homogeneous strata. The MAP prior approach is used to obtain stratum‐specific MAP prior and derive the overall propensity score integrated meta‐analytic predictive (PS‐MAP) prior. Additionally, we allow for tuning the prior effective sample size for the proposed PS‐MAP prior, which quantifies the amount of information borrowed from external data. We evaluate the performance of the proposed PS‐MAP prior by comparing it to the existing propensity score‐integrated power prior approach in a simulation study and illustrate its implementation with an example of a single‐arm phase II trial.

No abstract