Multiscale analysis approach for novelty detection in adaptation plot

Abstract

Recently, a multipurpose adaptive technique of evaluation of complicated behaviour of complex dynamic systems was introduced. That cognitive signal processing technique is based on evaluation and visualisation of unusual weight increments of sample-by-sample gradient descent adapted models. It has appeared that important attributes of complicated behaviour of complex dynamical systems can be captured and evaluated via much simpler (lower-dimensional) adaptive predictors. The visualisation method is called an Adaptation Plot (AP). So far, the weakness of AP is that AP needs manual tunning to reveal the important and otherwise hidden system (signal) attributes that emerge for a specific single setup of parameter. This paper introduces a multiscale approach for detection sensitivity of AP to overcome its single-setup limited interpretability.

Another Adaptive Approach to Novelty Detection in Time Series

“…The method presented in this paper is different from other methods and from the methods in [20] & [21], because this method is using actual error of the predictive model, together with cognitive information in changes of adaptive weights of the model. This is an interesting approach, because the prediction accuracy and behaviour of a learning model are not necessarily correlated with each other [21]; however, both values together provide us with important information about consistency at every sample with temporary system dynamics. These facts are the reason why this and similar models of novelty detection are important for future research and development.…”

“…Another approach to novelty detection is based on utilization of adaptive parameters of incrementally learning models (neural networks), i.e. the Adaptation Plot [20] that has been recently enhanced with multi-scale approach [21]. A most recent method is the Learning Entropy, i.e., a multi-scale approach to evaluation of unusual behaviour of adaptive parameters of a learning model is introduced in [22].…”

“…This paper however introduces another, different approach to novelty detection, that is neither based on statistical approaches, nor is it based on evaluation of error residual. Contrary to [21], this new approach operates only on parameter space of incrementally learning systems, and it does not use the multi-scale approach for detection sensitivity. Here the method introduced shows one of the possibilities how to detect perturbations within measured data in every new sample.…”

Learning Entropy: Multiscale Measure for Incremental Learning

“…It was shown through [1,2,[38][39][40] that low-dimensional predictors can capture and evaluate important signal attributes. As such, unusual samples, very decent perturbations, unusual appearance or variations of level of chaos or noise, incoming inter-attractor transitions of hyper-chaotic systems, also hidden repeating patterns can be revealed and intervals of a similar level of chaos can be revealed in otherwise seemingly, similarly complicated signals.…”

“…This paper demonstrates the novel approach on Gradient Descent (GD) adaptation that is one of the most comprehensible incremental learning techniques. The very original and funding principals and some related results with Adaptation Plot (AP) have been published in [1,2,38,39] and the first multi-scale extension was proposed in [40]; those are the funding concepts of Learning Entropy (LE) and the Approximate Individual Sample Learning Entropy (AISLE) that are introduced in this paper.…”