Very Fast Decision Rules for multi-class problems

Kosina, Petr; Gama, João

doi:10.1145/2245276.2245431

Cited by 15 publications

(10 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Alternative approaches, such as NIP-H e NIP-N, use Gaussian approximations instead of Hoeffding bounds in order to compute confidence intervals. Several extensions of VFDT have been proposed, also taking into account non-stationary data sources -see, e.g., [10], [9], [2], [35], [27], [15], [19], [21], [11], [34], [20], [29], [8]. All these methods are based on the classical Hoeffding bound [14]: after m independent observations of a random variable taking values in a real interval of size R, with probability at least 1 − δ the true mean does not differ from the sample mean by more than…”

Section: Introductionmentioning

confidence: 99%

Confidence Decision Trees via Online and Active Learning for Streaming Data

Rosa¹,

Cesa-Bianchi²

2017

jair

View full text Add to dashboard Cite

Decision tree classifiers are a widely used tool in data stream mining. The use of confidence intervals to estimate the gain associated with each split leads to very effective methods, like the popular Hoeffding tree algorithm. From a statistical viewpoint, the analysis of decision tree classifiers in a streaming setting requires knowing when enough new information has been collected to justify splitting a leaf. Although some of the issues in the statistical analysis of Hoeffding trees have been already clarified, a general and rigorous study of confidence intervals for splitting criteria is missing. We fill this gap by deriving accurate confidence intervals to estimate the splitting gain in decision tree learning with respect to three criteria: entropy, Gini index, and a third index proposed by Kearns and Mansour. Our confidence intervals depend in a more detailed way on the tree parameters. We also extend our confidence analysis to a selective sampling setting, in which the decision tree learner adaptively decides which labels to query in the stream. We furnish theoretical guarantee bounding the probability that the classification is nonoptimal learning the decision tree via our selective sampling strategy. Experiments on real and synthetic data in a streaming setting show that our trees are indeed more accurate than trees with the same number of leaves generated by other techniques and our active learning module permits to save labeling cost. In addition, comparing our labeling strategy with recent methods, we show that our approach is more robust and consistent respect all the other techniques applied to incremental decision trees. 1

show abstract

Section: Introductionmentioning

confidence: 99%

Confidence Decision Trees via Online and Active Learning for Streaming Data

Rosa¹,

Cesa-Bianchi²

2017

jair

View full text Add to dashboard Cite

show abstract

“…Nowadays, a variety of computational systems create enormous amounts of data, mostly in sequential fashion, and impose several constraints on available processing time and memory space. Extracting interesting patterns from data streams has received growing attention of the data mining community in the last few years [2,16,4,17].…”

Section: Data Stream Miningmentioning

confidence: 99%

A survey on feature drift adaptation: Definition, benchmark, challenges and future directions

Barddal

Gomes

Enembreck

et al. 2017

Journal of Systems and Software

View full text Add to dashboard Cite

Data stream mining is a fast growing research topic due to the ubiquity of data in several real-world problems. Given their ephemeral nature, data stream sources are expected to undergo changes in data distribution, a phenomenon called concept drift. This paper focuses on one specific type of drift that has not yet been thoroughly studied, namely feature drift. Feature drift occurs whenever a subset of features becomes, or ceases to be, relevant to the learning task, thus, learners must detect and adapt to these changes accordingly. We survey existing work on feature drift adaptation in both explicit and implicit approaches. Additionally, we benchmark several algorithms and a naive proposal in synthetic and real-world datasets. The results from our experiments indicate the need for future research in this area as even naive approaches produced gains in accuracy while reducing resources usage. Finally, we state current research topics, challenges and future directions for feature drift adaptation.

show abstract

“…Since this abundant -however raw -data do not provide interesting behavior patterns, data mining techniques, especially inductive learning, have been applied to extract useful knowledge from this type of data [13,10,21,6]. Extracting patterns from data streams and their usage in real-time is an effervescent research topic that has been tackled during the last decades [14,27,4,15,28].…”

Section: Learning From Data Streamsmentioning

confidence: 99%