Random forests: from early developments to recent advancements

Abstract

Ensemble classification is a data mining approach that utilizes a number of classifiers that work together in order to identify the class label for unlabeled instances. Random forest (RF) is an ensemble classification approach that has proved its high accuracy and superiority. With one common goal in mind, RF has recently received considerable attention from the research community to further boost its performance. In this paper, we look at developments of RF from birth to present. The main aim is to describe the research done to date and also identify potential and future developments to RF. Our approach in this review paper is to take a historical view on the development of this notably successful classification technique. We start with developments that were found before Breiman's introduction of the technique in 2001, by which RF has borrowed some of its components. We then delve into dealing with the main technique proposed by Breiman. A number of developments to enhance the original technique are then presented and summarized. Successful applications that utilized RF are discussed, before a discussion of possible directions of research is finally given.

Integrating Airborne Hyperspectral, Topographic, and Soil Data for Estimating Pasture Quality Using Recursive Feature Elimination with Random Forest Regression

“…During the process of elimination, 10-fold cross validation was implemented to optimize the variable selection and to ascertain the standard RF is a collection of several decision trees where each tree is constructed independently with random samples (n) from the training data. Random samples were drawn with the replacement from the training data using a bootstrap aggregating (bagging) method, which was found to be a more robust method for obtaining a stable model and helped to avoid overfitting [33]. Usually, 64% of training data is selected as in-bag data, and the remaining 36% were referred to as out-of-bag (OOB) data.…”

A fine-grained Random Forests using class decomposition: an application to medical diagnosis

“…Extensive experimental study has shown the potential of this approach. For more information about these techniques, the reader is referred to the survey paper in [13]. In a more recent work, diversification using weighted random subspacing was proposed in [12].…”

Combining Spectral and Texture Features Using Random Forest Algorithm: Extracting Impervious Surface Area in Wuhan

“…Combining Breiman's bagging sampling method (1996a) and the random selection of features introduced individually by Ho (1998) and Amit and Geman (1997), it perform excellently for linear and nonlinear prediction by keeping the balance between bias and variance. Additionally, the advantages of RF include (Fawagreh et al, 2014): 1. Accuracy of classification is very high 2.…”