Pruning and Nonparametric Multiple Change Point Detection

Zhang, Wenyu; James, Nicholas A.; Matteson, David S.

doi:10.1109/icdmw.2017.44

Cited by 21 publications

(15 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Note that, we also investigated other changepoint algorithms such as binary segmentation and segment neighborhoods algorithms but PELT was selected for its superior performance. Similar to the segment neighborhood algorithm, PELT also provides an exact segmentation but due to its dynamic programming which incorporates a pruning step it has been shown to be more computationally efficient, resulting in substantial reduction in computational time …”

Section: Methodsmentioning

confidence: 99%

“…Similar to the segment neighborhood algorithm, PELT also provides an exact segmentation but due to its dynamic programming which incorporates a pruning step it has been shown to be more computationally efficient, resulting in substantial reduction in computational time. [46][47][48][49][50] PELT employs a common approach of detecting changepoints through minimizing a cost function. For an ordered sequence of data z 1:n = (z 1 , … , z n ), PELT identifies s changepoints, 1:s = ( 1 , … , s ) that split the data into s + 1 segments with the ith segment containing z i−1 +1∶ i .…”

Section: Selection Threshold Optimized Empirically Via Splitting (Stomentioning

confidence: 99%

See 1 more Smart Citation

Optimized variable selection via repeated data splitting

Capanu

Giurcanu

Begg

et al. 2020

Statistics in Medicine

View full text Add to dashboard Cite

Model selection in high-dimensional settings has received substantial attention in recent years, however, similar advancements in the low-dimensional setting have been lacking. In this article, we introduce a new variable selection procedure for low to moderate scale regressions (n > p). This method repeatedly splits the data into two sets, one for estimation and one for validation, to obtain an empirically optimized threshold which is then used to screen for variables to include in the final model. In an extensive simulation study, we show that the proposed variable selection technique enjoys superior performance compared with candidate methods (backward elimination via repeated data splitting, univariate screening at 0.05 level, adaptive LASSO, SCAD), being amongst those with the lowest inclusion of noisy predictors while having the highest power to detect the correct model and being unaffected by correlations among the predictors. We illustrate the methods by applying them to a cohort of patients undergoing hepatectomy at our institution. K E Y W O R D Sdata splitting, empirical threshold, linear regression, variable screening, variable selection INTRODUCTIONRegression models have become the primary engine for many data analyses, attempting to explain the variability in the response variable by identifying important predictors. Due to their popularity, variable selection for multivariable regression continues to be one of the most important problems in applied statistics. An ideal variable selection method would include all "true" predictors in the model while excluding the "noisy" predictors, that is, those with little or no association with the response variable. These two objectives correspond to controlling the type II and type I errors in hypothesis testing. As such, there is a tradeoff between maximizing selection of true predictors (full model) and minimizing selection of noisy predictors (intercept only model) and a good variable selection procedure has to strike a balance between, with the goal of obtaining an accurate yet parsimonious model. A related way of thinking about this problem is the bias-variance tradeoff that comes with model complexity: more complex models will have higher variance and lower bias, while simpler models will have higher bias and lower variance. 1 Since the expected prediction error on new data can be decomposed into a bias component, a variance component, and noise (which is beyond our control), by choosing the model complexity to trade bias off with variance we, in effect, minimize the test error. This is a desirable goal for a model selection procedure since we want the model to predict future observations as well as it predicts the observed data.Many variable selection approaches have been proposed over the years including significance filtering, stepwise methods and penalized regression (some reviews can be found in References 1-3, among others). After the Statistics in Medicine. 2020;39:2167-2184.wileyonlinelibrary.com/journal/sim

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Selection Threshold Optimized Empirically Via Splitting (Stomentioning

confidence: 99%

Optimized variable selection via repeated data splitting

Capanu

Giurcanu

Begg

et al. 2020

Statistics in Medicine

View full text Add to dashboard Cite

show abstract

“…They are often associated with a likelihood or a ratio. Models may be distance based, such as similarity and dissimilarity distance [49], [50], fuzzy detection [51], kernel based rule methods and likelihood ratio methods.…”

Section: A Supervised Modelsmentioning

confidence: 99%

Online change detection techniques in time series: An overview

Namoano

Starr

Emmanouilidis

et al. 2019

2019 IEEE International Conference on Prognostics and Health Management (ICPHM)

View full text Add to dashboard Cite

Time-series change detection has been studied in several fields. From sensor data, engineering systems, medical diagnosis, and financial markets to user actions on a network, huge amounts of temporal data are generated. There is a need for a clear separation between normal and abnormal behaviour of the system in order to investigate causes or forecast change. Characteristics include irregularities, deviations, anomalies, outliers, novelties or surprising patterns. The efficient detection of such patterns is challenging, especially when constraints need to be taken into account, such as the data velocity, volume, limited time for reacting to events, and the details of the temporal sequence. This paper reviews the main techniques for time series change point detection, focusing on online methods. Performance criteria including complexity, time granularity, and robustness is used to compare techniques, followed by a discussion about current challenges and open issues.

show abstract

“…The motivation of this chapter is to describe a new approach to change-point detection in time series and demonstrate its application in the context of COVID-19 pandemic (analysis of geolocation tracking data, detection of changes in health data of patient etc.). Now, one of the most popular online statistical methods for solving such problems is the Kolmogorov-Smirnov test [6][7][8][9][10]. This non-parametric test is simple and effective when samples are not overlapping or they have small overlapping.…”

Section: Introductionmentioning

confidence: 99%

Nonparametric Analysis of Tracking Data in the Context of COVID-19 Pandemic

Klyushin

2020

Studies in Big Data

View full text Add to dashboard Cite

Methods of statistical pattern recognition are powerful tools for analysis of statistical data on COVID-19 pandemic. In this chapter, we offer a new effective online algorithm for detection of change-points in tracking data (movement data, health rate data etc.). We developed a non-parametric test for evaluation of the statistical hypothesis that data in two adjacent time intervals have the same distribution. In the context of the mobile phone tracking this means that the coordinates of the tracked object does not deviated from the base point significantly. For estimation of the health rate it means the absence of significant deviations from the norm. The significance level for the test is less than 0.05. The test permits ties in samples. Also, we show that results of comparison of the test with well-known Kolmogorov-Smirnov test. These results show that the proposed test is more robust, sensitive and accurate than the alternative ones. In addition, the new method does not require high computational capacity.

show abstract

Pruning and Nonparametric Multiple Change Point Detection

Cited by 21 publications

References 22 publications

Optimized variable selection via repeated data splitting

Optimized variable selection via repeated data splitting

Online change detection techniques in time series: An overview

Nonparametric Analysis of Tracking Data in the Context of COVID-19 Pandemic

Contact Info

Product

Resources

About