Forest trees for on-line data

Abstract

This paper presents an hybrid adaptive system for induction of forest of trees from data streams. The Ultra Fast Forest Tree system (UFFT) is an incremental algorithm, with constant time for processing each example, works online, and uses the Hoeffding bound to decide when to install a splitting test in a leaf leading to a decision node. Our system has been designed for continuous data. It uses analytical techniques to choose the splitting criteria, and the information gain to estimate the merit of each possible splitting-test. The number of examples required to evaluate the splitting criteria is sound, based on the Hoeffding bound. For multiclass problems,the algorithm builds a binary tree for each possible pair of classes, leading to a forest of trees. During the training phase the algorithm maintains a short term memory. Given a data stream, a fixed number of the most recent examples are maintained in a data-structure that supports constant time insertion and deletion. When a test is installed, a leaf is transformed into a decision node with two descendant leaves. The sufficient statistics of these leaves are initialized with the examples in the short term memory that will fall at these leaves. We study the behavior of UFFT in different problems. The experimental results shows that UFFT is competitive against a batch decision tree learner in large and medium datasets.

Stress-Testing Hoeffding Trees

“…These results contradict those reported by Gama et al [3,4], whose conclusion was that Naive Bayes leaves are always better on the LED data. They used a hold out test set and quoted the final accuracy attained.…”

“…One solution has been proposed by Gama who in [3] suggests the use of a short term memory cache housing some of the recently seen instances. A problem with this solution is determining a sufficient size for the cache.…”

“…Standard Hoeffding trees use the majority class at each leaf for prediction. Previous work [3] has shown that adding socalled functional (or Naive Bayes) leaves to Hoeffding trees for both synthetically generated streams, and real datasets of sufficient size, as well as in the presence of noise outperforms standard Hoeffding trees.…”

Improving Adaptive Bagging Methods for Evolving Data Streams

“…A significant increase in that rate indicates that the data is changing with respect to the time at which the subtree was created. This was the approach used by Gama et al in [12], using another change detector.…”

Data streams classification by incremental rule learning with parameterized generalization

“…Recent works on data streams classification has been mainly addressed by two different approaches: decision trees [2,6,7] and ensemble methods [16,18].…”