Extending Statistical Boosting

Abstract

SummaryBackground: Boosting algorithms to simultaneously estimate and select predictor effects in statistical models have gained substantial interest during the last decade. Objectives: This review highlights recent methodological developments regarding boosting algorithms for statistical modelling especially focusing on topics relevant for biomedical research. Methods: We suggest a unified framework for gradient boosting and likelihood-based boosting (statistical boosting) which have been addressed separately in the literature up to now.

Generating highly accurate prediction hypotheses through collaborative ensemble learning

“…The ensemble superscript ( j ) might be left out in the following text for brevity, especially when talking about a single Gentle Boost ensemble whose position within the multi-model is unimportant. Other methods which could be incorporated in our collaborative strategy include gradient boosting and likelihood-based boosting (statistical boosting) algorithms37, as well as algorithms that optimize non-convex potential functions instead of the traditional exponential loss function4445.…”

“…Boosting algorithms can be modified such that they contain an intrinsic mechanism for variable selection and model choice (component-wise learning35). Recently, Mayr et al 37. provide comprehensive overviews on the evolution of boosting algorithms, as well as on extending statistical boosting.…”

Predictive Modelling Based on Statistical Learning in Biomedicine

“…In doing so, they update and expand earlier reviews of this specific area of statistical learning research [4–6]. For the first time, recent methodologic research on boosting functional data and on the application of boosting techniques in advanced survival modelling is reviewed.…”

An Update on Statistical Boosting in Biomedicine

“…Likelihood-based boosting [ 3 , 29 ] is the other general approach in the framework of statistical boosting algorithms; it received much attention particularly in the context of high-dimensional biomedical data (see [ 11 ] and the references therein). Although it follows a very similar structure to gradient boosting (see Box 1 ), both approaches only coincide in special cases such as classical Gaussian regression via the L 2 loss [ 1 , 30 ].…”

“…An accompanying article [ 11 ] highlighted the multiple extension of the basic algorithms towards (i) enhanced variable selection properties, (ii) new types of predictor effects, and (iii) new regression settings. Substantial methodological developments on statistical boosting algorithms throughout the last few years (e.g., stability selection [ 12 ]) and a growing community have opened the door to new model classes and frameworks (e.g., joint models [ 13 ] and functional data [ 14 ]), asking for an up-to-date review on the available extensions.…”