Model Change Detection With the MDL Principle

Yamanishi, Kenji; Fukushima, Shintaro

doi:10.1109/tit.2018.2852747

Cited by 19 publications

(23 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the case where the true parameter values are unknown, the MDL change statistics has been proposed to measure the change degree 14 , 18 from a given data sequence. Below we denote .…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Change sign detection with differential MDL change statistics and its applications to COVID-19 pandemic analysis

Yamanishi

Yuki

et al. 2021

Sci Rep

Self Cite

View full text Add to dashboard Cite

We are concerned with the issue of detecting changes and their signs from a data stream. For example, when given time series of COVID-19 cases in a region, we may raise early warning signals of an epidemic by detecting signs of changes in the data. We propose a novel methodology to address this issue. The key idea is to employ a new information-theoretic notion, which we call the differential minimum description length change statistics (D-MDL), for measuring the scores of change sign. We first give a fundamental theory for D-MDL. We then demonstrate its effectiveness using synthetic datasets. We apply it to detecting early warning signals of the COVID-19 epidemic using time series of the cases for individual countries. We empirically demonstrate that D-MDL is able to raise early warning signals of events such as significant increase/decrease of cases. Remarkably, for about $$64\%$$ 64 % of the events of significant increase of cases in studied countries, our method can detect warning signals as early as nearly six days on average before the events, buying considerably long time for making responses. We further relate the warning signals to the dynamics of the basic reproduction number R0 and the timing of social distancing. The results show that our method is a promising approach to the epidemic analysis from a data science viewpoint.

show abstract

“…In the case where the true parameter values are unknown, the MDL change statistics has been proposed to measure the change degree 14 , 18 from a given data sequence. Below we denote .…”

Section: Methodsmentioning

confidence: 99%

“…The MDL change statistics has been proposed as a test statistics in the hypothesis testing for change detection 14 , 18 . It is defined as the difference between the total codelength required for encoding data for the non-change case and that for the change case at a specific time point t .…”

Section: Introductionmentioning

confidence: 99%

Change sign detection with differential MDL change statistics and its applications to COVID-19 pandemic analysis

Yamanishi

Yuki

et al. 2021

Sci Rep

Self Cite

View full text Add to dashboard Cite

show abstract

“…In particular, in [ 37 ], it has been observed that prequential coding yields much better codelengths than variational inference, correlating better with the test set performance — we remind that in the prequential coding, a model with default values is used to encode the first few data; then, the model is trained on these few encoded data; the partially trained model is used to encode the next data; then, the model is retrained on all data encoded so far, and so on. On the contrary, in [ 38 ], an MDL-based strategy is used for determining a parameter-free stopping criterion for semi-supervised learning in time series classification, while in [ 39 ], the problem of model change tracking and detection has been addressed and studied in both data-compression and hypothesis-testing scenarios. In the first case, an upper bound for the minimax regret for model changes has been found; in the second one, error probabilities for the MDL change test have been derived, and they rely on the information-theoretic complexity, i.e., the complexity of the model class or the model itself and the

-divergence.…”

Section: MDL Applications: a Reviewmentioning

confidence: 99%

A Short Review on Minimum Description Length: An Application to Dimension Reduction in PCA

Bruni

Cardinali

Vitulano

2022

Entropy

View full text Add to dashboard Cite

The minimun description length (MDL) is a powerful criterion for model selection that is gaining increasing interest from both theorists and practicioners. It allows for automatic selection of the best model for representing data without having a priori information about them. It simply uses both data and model complexity, selecting the model that provides the least coding length among a predefined set of models. In this paper, we briefly review the basic ideas underlying the MDL criterion and its applications in different fields, with particular reference to the dimension reduction problem. As an example, the role of MDL in the selection of the best principal components in the well known PCA is investigated.

show abstract

“…That is, learning and change detection are conducted by finding the probability distributions with the shortest description lengths. The key idea of our framework is to combine the MDL change statistics [11], [12] with the DNML (decomposed normalized maximum likelihood) codelength [13], [14] calculation method in change scoring for latent variable models. First, we employ the MDL change statistics to score the degree of change at any given time point.…”

Section: B Novelty and Significancementioning

confidence: 99%

Detecting Hierarchical Changes in Latent Variable Models

Fukushima

Yamanishi

2020

2020 IEEE International Conference on Data Mining (ICDM)

Self Cite

View full text Add to dashboard Cite

This paper addresses the issue of detecting hierarchical changes in latent variable models (HCDL) from data streams. There are three different levels of changes for latent variable models: 1) the first level is the change in data distribution for fixed latent variables, 2) the second one is that in the distribution over latent variables, and 3) the third one is that in the number of latent variables. It is important to detect these changes because we can analyze the causes of changes by identifying which level a change comes from (change interpretability). This paper proposes an information-theoretic framework for detecting changes of the three levels in a hierarchical way. The key idea to realize it is to employ the MDL (minimum description length) change statistics for measuring the degree of change, in combination with DNML (decomposed normalized maximum likelihood) code-length calculation. We give a theoretical basis for making reliable alarms for changes. Focusing on stochastic block models, we employ synthetic and benchmark datasets to empirically demonstrate the effectiveness of our framework in terms of change interpretability as well as change detection.

show abstract

Model Change Detection With the MDL Principle

Cited by 19 publications

References 31 publications

Change sign detection with differential MDL change statistics and its applications to COVID-19 pandemic analysis

Change sign detection with differential MDL change statistics and its applications to COVID-19 pandemic analysis

A Short Review on Minimum Description Length: An Application to Dimension Reduction in PCA

Detecting Hierarchical Changes in Latent Variable Models

Contact Info

Product

Resources

About