Online Anomaly Detection With Nested Trees

Abstract

We introduce an online anomaly detection algorithm that processes data in a sequential manner. At each time, the algorithm makes a new observation, produces a decision, and then adaptively updates all its parameters to enhance its performance. The algorithm mainly works in an unsupervised manner since in most real-life applications labeling the data is costly. Even so, whenever there is a feedback, the algorithm uses it for better adaptation. The algorithm has two stages. In the first stage, it constructs a score function similar to a probability density function to model the underlying nominal distribution (if there is one) or to fit to the observed data. In the second state, this score function is used to evaluate the newly observed data to provide the final decision. The decision is given after the well-known thresholding. We construct the score using a highly versatile and completely adaptive nested decision tree. Nested soft decision trees are used to partition the observation space in a hierarchical manner. We adaptively optimize every component of the tree, i.e., decision regions and probabilistic models at each node as well as the overall structure, based on the sequential performance. This extensive in-time adaptation provides strong modeling capabilities; however, it may cause overfitting. To mitigate the overfitting issues, we first use the intermediate nodes of the tree to produce several subtrees, which constitute all the models from coarser to full extend, and then adaptively combine them. By using a real-life dataset, we show that our algorithm significantly outperforms the state of the art.

Optimally Efficient Sequential Calibration of Binary Classifiers to Minimize Classification Error

“…In most prominent detection, estimation, prediction and learning problems [1], [2], intelligent agents often make decisions under considerable uncertainty (randomness, noise, incomplete data), where they combine features to determine the actions that maximize some utility [3]. The applications are numerous in many fields including decision theory [4], control theory [5], game theory [6], [7], optimization [8], [9], distribution estimation [10]- [13], anomaly detection [14], [15], signal processing [16], [17], prediction [18], [19] and bandits [20], [21]. The outputs of these learning models are designed to discriminate the data patterns and provide accurate probabilities for practical usefulness.…”

A Hybrid Modeling of Mobile App Dynamics on Serial Causality for Malware Detection

“…Machine learning-based detection methods mainly include support vector machines [16], random forests [17], and decision trees [18]. Narudin et al [19] propose a solution for malware detection that uses machine learning to evaluate various network traffic characteristics.…”

Efficient, Anytime Algorithms for Calibration with Isotonic Regression under Strictly Convex Losses

“…In the problems of learning, recognition, estimation or prediction [1]- [3]; decisions are often produced to minimize certain loss functions using features of the observations, which are generally noisy, random or even missing. There are numerous applications in a number of varying fields such as decision theory [4], control theory [5], game theory [6], [7], optimization [8], [9], density estimation and anomaly detection [10]- [15], scheduling [16], signal processing [17], [18], forecasting [19], [20] and bandits [21]- [23]. These decisions are acquired from specific learning models, where the goal is to distinguish certain data patterns and provide accurate estimations for practical use.…”