Federated Ensemble Regression Using Classification

Abstract

Ensemble learning has been shown to significantly improve predictive accuracy in a variety of machine learning problems. For a given predictive task, the goal of ensemble learning is to improve predictive accuracy by combining the predictive power of multiple models. In this paper, we present an ensemble learning algorithm for regression problems which leverages the distribution of the samples in a learning set to achieve improved performance. We apply the proposed algorithm to a problem in precision medicine where the goal is to predict drug perturbation effects on genes in cancer cell lines. The proposed approach significantly outperforms the base case.

Imbalanced regression using regressor-classifier ensembles

“…Implicit in this implementation is the assumption that one will use upsampling to resolve class imbalance issues. Empirical evidence from our previous work indicates that this is necessary, where we found that upsampling outperformed all other methods for handling class imbalance in this setting (Orhobor et al, 2020). However, replacing it with one's preferred method is trivial.…”

“…Finally, these combined predictions are aggregated using weights that are determined on the learning set. The intuition is that this process should leverage the predictive accuracy of the base and binned models, significantly reducing the possibility of performing worse than the base case, irrespective of the choice of discretizer, which improves upon our previous work (Orhobor et al, 2020). The proposed algorithm is described in Fig.…”

“…Orhobor et al, 2020. Learning set matrix L ∈ IR m×d , response vector y, s maximum bin size, k crossvalidation folds, and test set matrix T ∈ IR n×d Output: Weighted test set predictions and best performing individual bin size predictions Validate maximum bin size: 1: Validate maximum bin size s. Randomly split y into k folds. Discretize each y k , where bin size is set to s. For some discretizers and y k , the maximum number of possible bins is less than s, set s to the minimum of bins from discretizing all y k .…”

Early Detection of Alzheimer’s Disease: An Extensive Review of Advancements in Machine Learning Mechanisms Using an Ensemble and Deep Learning Technique

“…On the other hand, Random Forest achieves high accuracy predicting AD using limited features from MRI scans [17]. Deep learning has shown potential in AD diagnoses, especially when studying complex disease pathways [18]. Still, its reliability in predicting AD progression needs rigorous testing across various imaging modalities and larger datasets.…”

“…Other studies have used advanced machine learning methods such as deep learning and convolutional neural networks for AD diagnoses [15,16]. Novel biomarkers and multimodal data integration [17,18] have shown promise in predicting AD. This study utilizes a robust AD dataset from UCI's machine learning repository, ensuring findings' reliability and generalizability.…”