Online event-driven subsequence matching over financial data streams

Abstract

Subsequence similarity matching in time series databases is an important research area for many applications. This paper presents a new approximate approach for automatic online subsequence similarity matching over massive data streams. With a simultaneous online segmentation and pruning algorithm over the incoming stream, the resulting piecewise linear representation of the data stream features high sensitivity and accuracy. The similarity definition is based on a permutation followed by a metric distance function, which provides the similarity search with flexibility, sensitivity and scalability. Also, the metric-based indexing methods can be applied for speed-up. To reduce the system burden, the event-driven similarity search is performed only when there is a potential event. The query sequence is the most recent subsequence of piecewise data representation of the incoming stream which is automatically generated by the system. The retrieved results can be analyzed in different ways according to the requirements of specific applications. This paper discusses an application for future data movement prediction based on statistical information. Experiments on real stock data are performed. The correctness of trend predictions is used to evaluate the performance of subsequence similarity matching.

Contour map matching for event detection in sensor networks

“…Finally, the detection of the common types of events that we define bears some similarity to the shape matching queries in traditional [2] or streaming [28] time series databases. Since our work is targeted at sensor network surveillance applications, we focus on enabling the detection of these events in a simple but effective way via in-network contour mapping.…”

“…Now that we convert the event detection problem into a pattern matching one, the next question is how to solve the pattern matching problem in sensor networks effectively and efficiently. There has been a great wealth of literature on pattern matching [2] [28], but the challenge is to seek a solution that works for a resource-limited network in a distributed, real-time, and energyefficient way. The solution we find is in contour maps of sensory data distribution.…”

Online piece-wise linear approximation of numerical streams with precision guarantees

“…They do not provide precision guarantees, but rather protect the servers from overwhelming data rates. Wu et al [24] consider the approximation of financial data streams, where the data follows a repetitive pattern of waves. Therefore, the piece-wise linear approximation generated by their algorithm has a zigzag shape.…”

“…Palpanas et al [22] introduced the amnesic approximation of data streams, which allows arbitrary, user defined reduction of quality with time. The work in both [24] and [22] does not provide precision guarantees either. Keogh et al [16] proposed the SWAB algorithm which merges an offline bottom-up technique for time series segmentation with an online technique similar to the linear filter.…”

Pattern discovery in data streams under the time warping distance

“…Data streams are becoming increasingly important in several domains including financial data analysis [68], sensor network monitoring [74], moving object trajectories [15,38], web click-stream analysis [42,48,59], and network traffic analysis [37]. Many applications require time-series data streams to be continuously monitored in real time, and the processing and mining of data streams are attracting increasing interest.…”