Weighted Random Forests to Improve Arrhythmia Classification

Abstract

Construction of an ensemble model is a process of combining many diverse base predictive learners. It arises questions of how to weight each model and how to tune the parameters of the weighting process. The most straightforward approach is simply to average the base models. However, numerous studies have shown that a weighted ensemble can provide superior prediction results to a simple average of models. The main goals of this article are to propose a new weighting algorithm applicable for each tree in the Random Forest model and the comprehensive examination of the optimal parameter tuning. Importantly, the approach is motivated by its flexibility, good performance, stability, and resistance to overfitting. The proposed scheme is examined and evaluated on the Physionet/Computing in Cardiology Challenge 2015 data set. It consists of signals (electrocardiograms and pulsatory waveforms) from intensive care patients which triggered an alarm for five cardiac arrhythmia types (Asystole, Bradycardia, Tachycardia, Ventricular Tachycardia, and Ventricular Fultter/Fibrillation). The classification problem regards whether the alarm should or should not have been generated. It was proved that the proposed weighting approach improved classification accuracy for the three most challenging out of the five investigated arrhythmias comparing to the standard Random Forest model.

Weighted Quantile Regression Forests for Bimodal Distribution Modeling: A Loss Given Default Case

“…The Random Forest algorithm introduced by Breiman [ 24 ] works by constructing many decision trees and outputting the prediction of the individual trees by utilizing a training sample (training dataset ) of observations, target variables (where is an empirical realization of ) and predictor variables , where each feature can take a value from its own set of possible values (moreover, is an empirical realization of ). The main objective of each decision tree is to find a model ( ) for predicting the values of from new values [ 7 ]. In theory, the solution is simply a recursive partition of space into the disjointed rectangular subspace of (eventually achieving a final node called the leaf, denoted as ) in such a way that the predicted ( ) value of minimizes the total impurity of its child nodes (it is usually assumed that each parent node has two children, i.e., binary tree are considered).…”

“…In theory, the solution is simply a recursive partition of space into the disjointed rectangular subspace of (eventually achieving a final node called the leaf, denoted as ) in such a way that the predicted ( ) value of minimizes the total impurity of its child nodes (it is usually assumed that each parent node has two children, i.e., binary tree are considered). One of the first and widely used decision tree algorithms is the classification and regression tree (CART) [ 58 ], employing a measure of node impurity based on the distribution of the observed values in the node by splitting a node that minimizes the total impurity of its two child nodes, defined by the total sum of squares [ 7 ]: where denotes the average value of vector over all observations belonging to a particular node. The process is applied recursively to the data in each child node.…”

“…There are various stopping criteria controlling the growth of the tree such as the minimum number of observations that must exist in a node in order for a split to be attempted, the minimum number of observations in any terminal node (leaf), or the maximum depth of any node of the final tree [ 7 ]. To achieve good generalization ability (i.e., small error rate on the unseen examples), the tree first grows in an overly large size and then it is pruned to a smaller size to minimize the misclassification error.…”

“…Alongside such techniques like bagging, boosting and simple voting, one can employ the Random Forest algorithm (RF). The purpose of RF is to combine forecasts of several base models (decision trees) in order to improve reliability and generalizability over a single model [ 7 ]. There is strong evidence that a single model can be outperformed by an ensemble of models combined to reduce bias, variance or both.…”

“…The simplest approach is to give equal weight to each learner. However, numerous studies have shown that a weighted ensemble can provide better forecasting results [ 7 ]. For this reason, this research presents a novel weighted quantile Regression Forest algorithm (wqRF), which assigns higher weights in quantile estimation to those base learners with better generalization abilities.…”