A WED Method for Evaluating the Performance of Change-Point Detection Algorithms

Abstract

Change point detection (CPD) is to find the abrupt changes in a time series. Various computational algorithms have been developed for CPD. To compare the different CPD models, many performance metrics have been introduced to evaluate the algorithms. Each of the previous evaluation methods measures the different aspect of the methods. In this paper, a new weighted error distance (WED) method is proposed to evaluate the overall performance of a CPD model across multiple time series of different lengths. A concept of normalized error distance was introduced to allow comparison of the distances between an estimated change point position and the target change point among models that work on multiple time series. In this study, the WED metrics was applied on synthetic datasets with different sample sizes and variances to evaluate the different CPD models, including: Kolmogorov-Smirnov (KS), SSA and T algorithms. The test results showed the value of this WED method that contributes to the methodology for evaluating the performance of CPD models.

A Weighted Error Distance Metrics (WEDM) for Performance Evaluation on Multiple Change-Point (MCP) Detection in Synthetic Time Series

“…In our previous studies [ 33 ], a preliminary WED method was proposed for evaluating a CPD model for single change-point detection. In this existing method, a concept of weighted error distance (WED) is introduced for counting a normalized error distance between each pair of the resultant eCPs and the actual tCPs, and then the performance of different CPD models is ranked by the averaged WED accordingly [ 33 ]. In this study, a novel WEDM method is proposed to compare the overall performance of CPD models for MCPs detection on multiple data segments in a time series with different data features.…”