Incremental multi-target model trees for data streams

Abstract

As in batch learning, one may identify a class of streaming real-world problems which require the modeling of several targets simultaneously. Due to the dependencies among the targets, simultaneous modeling can be more successful and informative than creating independent models for each target. As a result one may obtain a smaller model able to simultaneously explain the relations between the input attributes and the targets. This problem has not been addressed previously in the streaming setting. We propose an algorithm for inducing multi-target model trees with low computational complexity, based on the principles of predictive clustering trees and probability bounds for supporting splitting decisions. Linear models are computed for each target separately, by incremental training of perceptrons in the leaves of the tree. Experiments are performed on synthetic and real-world datasets. The multi-target regression tree algorithm produces equally accurate and smaller models for simultaneous prediction of all the target attributes, as compared to a set of independent regression trees built separately for each target attribute. When the regression surface is smooth, the linear models computed in the leaves significantly improve the accuracy for all of the targets.

A survey on multi‐output regression

“…Ikonomovska et al proposed an incremental multi‐target model tree algorithm, referred to as FIMT‐MT, for simultaneous modeling of multiple continuous targets from time changing data streams. FIMT‐MT extends an incremental single‐target model tree by adopting the principles of the predictive clustering methodology in the split selection criterion.…”

Scalable and efficient multi-label classification for evolving data streams

“…In a regression context, an arguably similar setting to multi-label learning is the multitarget problem, that is addressed in Appice and Džeroski (2007), and on data streams in Ikonomovska et al (2011). However, we are not aware of any existing work dealing with this approach on multi-label data streams; the work done in this paper does not deal with any multi-label data sources, and thus is it not of specific relevance to this paper.…”

Multi-label classification via multi-target regression on data streams

“…To utilize the MLC via MTR approach, we have reimplemented the FIMT and FIMT-MT algorithms (Ikonomovska et al 2011a) in the MOA framework to facilitate usability and visibility, as the original implementation was a standalone extension of the C-based VFML library (Hulten and Domingos 2003) and was not available as part of a larger data stream mining framework. We have also significantly extended the algorithm to consider nominal attributes in the input space when considering splitting decisions.…”

“…Ikonomovska et al (2011b) introduced an instance-incremental streaming tree-based singletarget regressor (FIMT-DD) that utilized the Hoeffding bound. This work was later extended to the multi-target regression setting (Ikonomovska et al 2011a) (FIMT-MT). There has been a theoretical debate on whether the use of the Hoeffding bound is appropriate (Rutkowski et al 2013), but, a recent study by Ikonomovska et al (2015) has shown that, in practice, the use of the Hoeffding bound produces good results.…”