Sequential Outlier Detection Based on Incremental Decision Trees

Abstract

We introduce an online outlier detection algorithm to detect outliers in a sequentially observed data stream. For this purpose, we use a two-stage filtering and hedging approach. In the first stage, we construct a multimodal probability density function to model the normal samples. In the second stage, given a new observation, we label it as an anomaly if the value of aforementioned density function is below a specified threshold at the newly observed point. In order to construct our multimodal density function, we use an incremental decision tree to construct a set of subspaces of the observation space. We train a single component density function of the exponential family using the observations, which fall inside each subspace represented on the tree. These single component density functions are then adaptively combined to produce our multimodal density function, which is shown to achieve the performance of the best convex combination of the density functions defined on the subspaces. As we observe more samples, our tree grows and produces more subspaces. As a result, our modeling power increases in time, while mitigating overfitting issues. In order to choose our threshold level to label the observations, we use an adaptive thresholding scheme. We show that our adaptive threshold level achieves the performance of the optimal prefixed threshold level, which knows the observation labels in hindsight. Our algorithm provides significant performance improvements over the state of the art in our wide set of experiments involving both synthetic as well as real data.

Regret Analysis of Global Optimization in Univariate Functions with Lipschitz Derivatives

“…• Bandit optimization over metric spaces [17], due to numerous applications in decision theory [18], control theory [19], game theory [20], distribution estimation [21]- [24], anomaly detection [25], [26], signal processing [27], prediction [28], [29] and bandits [30], [31].…”

Low Regret Binary Sampling Method for Efficient Global Optimization of Univariate Functions

“…Over the past years, the global optimization problem has gathered significant attention with various algorithms being proposed in distinct fields of research. It has been studied especially in the fields of non-convex optimization [6]- [8], Bayesian optimization [9], convex optimization [10]- [12], bandit optimization [13], stochastic optimization [14], [15]; because of its practical applications in distribution estimation [16]- [19], multi-armed bandits [20]- [22], control theory [23], signal processing [24], game theory [25], prediction [26], [27], decision theory [28] and anomaly detection [29]- [31].…”

Optimally Efficient Sequential Calibration of Binary Classifiers to Minimize Classification Error

“…In most prominent detection, estimation, prediction and learning problems [1], [2], intelligent agents often make decisions under considerable uncertainty (randomness, noise, incomplete data), where they combine features to determine the actions that maximize some utility [3]. The applications are numerous in many fields including decision theory [4], control theory [5], game theory [6], [7], optimization [8], [9], distribution estimation [10]- [13], anomaly detection [14], [15], signal processing [16], [17], prediction [18], [19] and bandits [20], [21]. The outputs of these learning models are designed to discriminate the data patterns and provide accurate probabilities for practical usefulness.…”