Stacked penalized logistic regression for selecting views in multi-view learning

Abstract

In biomedical research many different types of patient data can be collected, including various types of omics data and medical imaging modalities. Applying multi-view learning to these different sources of information can increase the accuracy of medical classification models compared with single-view procedures. However, the collection of biomedical data can be expensive and taxing on patients, so that superfluous data collection should be avoided. It is therefore necessary to develop multi-view learning methods which can accurately identify the views most important for prediction.In recent years, several biomedical studies have used an approach known as multi-view stacking (MVS), where a model is trained on each view separately and the resulting predictions are combined through stacking. In these studies, MVS has been shown to increase classification accuracy. However, the MVS framework can also be used for selecting a subset of important views.To study the view selection potential of MVS, we develop a special case called stacked penalized logistic regression (StaPLR). Compared with existing view-selection methods, StaPLR can make use of faster optimization algorithms and is easily parallelized. We show that nonnegativity constraints on the parameters of the function which combines the views are important for preventing unimportant views from entering the model. We investigate the performance of StaPLR through simulations, and consider two real data examples. We compare the performance of StaPLR with an existing view selection method called the group lasso and observe that, in terms of view selection, StaPLR has a consistently lower false positive rate.

SUBiNN: a stacked uni- and bivariate kNN sparse ensemble

“…Breiman (1996) suggested this non-negativity constraint on the coefficients of the regression meta-learner. Further support for this constraint is found in Leblanc and Tibshirani (1996) and Van Loon et al (2020). For the choice of an optimal λ value we use 10-fold cross-validation.…”

Analyzing hierarchical multi-view MRI data with StaPLR: An application to Alzheimer's disease classification

“…A general term for data in which the features are divided into feature sets (for example, by source or modality) is multi-view data, and the field of developing algorithms for such data is known as multi-view (machine) learning [12,13]. Of particular interest to this study is the multi-view learning framework known as multi-view stacking [11,14,15]. The general idea of multi-view stacking is to first train a model on each feature set (also called a view ) separately.…”

“…For example, if a certain scan modality turns out to be irrelevant for prediction of a disease, it may not have to be measured at all. Recently, a variant of multi-view stacking called stacked penalized logistic regression (StaPLR) has been developed specifically for this purpose [15]. StaPLR essentially integrates the penalized logistic regression models which are already commonly used in neuroimaging classification, such as ridge regression [17,18] and the lasso [19,20], into the multiview stacking methodology.…”

“…StaPLR essentially integrates the penalized logistic regression models which are already commonly used in neuroimaging classification, such as ridge regression [17,18] and the lasso [19,20], into the multiview stacking methodology. StaPLR can be used to select the feature sets that are most relevant for prediction, and has been shown to have several advantages over earlier methods, including a decreased false positive rate in view selection and a large reduction in computation time, while maintaining good classification accuracy [15]. In addition StaPLR is, to our knowledge, the only multi-view learning method which can easily be extended to a hierarchical multi-view structure with an arbitrary number of levels while keeping computational feasibility.…”

“…Previous applications of StaPLR have focused solely on a setting with two levels [15,23]. In this paper we will therefore addapt StaPLR to the hierarchical structure of the data.…”