A Simple Dimensionality Reduction Technique for Fast Similarity Search in Large Time Series Databases

Abstract

Abstract.We address the problem of similarity search in large time series databases. We introduce a novel-dimensionality reduction technique that supports an indexing algorithm that is more than an order of magnitude faster than the previous best known method. In addition to being much faster our approach has numerous other advantages. It is simple to understand and implement, allows more flexible distance measures including weighted Euclidean queries and the index can be built in linear time. We call our approach PCA-indexing (Piecewise Constant Approximation) and experimentally validate it on space telemetry, financial, astronomical, medical and synthetic data.

Anomaly Base Network Intrusion Detection by Using Random Decision Tree and Random Projection: A Fast Network Intrusion Detection Technique

“…Agrawal et al [17] map time series into frequency domain by the discrete Fourier transform and only retain the first few frequencies. Keogh and Pazzani [18] reduce the dimension of time series data by segmenting the time series into sections and indexing only the section means. Agarwal et al [19] index market basket data by a specific signature table, which easens the similarity search.…”

Assessing the Possibility of Identifying Precancerous Cervical Lesions Using Aceto-White Temporal Patterns

“…The reduction of dimensionality in the x-axis is obtained by dividing the total length of the time series into fragments of a certain size (word size). It is also necessary to establish a number of intervals in the y-axis to compress the values of the time series (size of the alphabet) (Keogh & Pazzani, 2000). In this case, the parameters of word size and alphabet were selected experimentally.…”

Pattern Discovery in Time-Ordered Data

“…This nonlinear alignment results in a more sophisticated distance measure as shown in Figure 2(B). There are numerous papers on using DTW for time series data, including [6] [7][8] [9][10]. Berndt and Clifford [6] describe a DTW algorithm as follows:…”

“…As a result, we explored techniques for pattern matching and pattern discovery in both continuous and discrete events. For pattern matching in continuous data, we studied the use of Dynamic Time Warping (DTW) [6] [7][8][9] [10] or Hidden Markov Models. We had a limited set of analog data to work with for this aspect of the project and were not able to proceed beyond the initial investigatory steps.…”

“…As a result, there were two major thrusts to the LDRD. The first aspect studied was the use of Dynamic Time Warping (DTW) [6] [7][8] [9][10] for pattern matching in continuous data. In essence, DTW is a technique for aligning time series along the time axis to optimize the similarity measure.…”