Online piece-wise linear approximation of numerical streams with precision guarantees

Abstract

Continuous "always-on" monitoring is beneficial for a number of applications, but potentially imposes a high load in terms of communication, storage and power consumption when a large number of variables need to be monitored. We introduce two new filtering techniques, swing filters and slide filters, that represent within a prescribed precision a time-varying numerical signal by a piecewise linear function, consisting of connected line segments for swing filters and (mostly) disconnected line segments for slide filters. We demonstrate the effectiveness of swing and slide filters in terms of their compression power by applying them to a reallife data set plus a variety of synthetic data sets. For nearly all combinations of signal behavior and precision requirements, the proposed techniques outperform the earlier approaches for online filtering in terms of data reduction. The slide filter, in particular, consistently dominates all other filters, with up to twofold improvement over the best of the previous techniques.

A Survey of Model-based Sensor Data Acquisition and Management

“…, K, approximates the data items (( , (t i k , v i k )). Similar to [20], we distinguish between two classes of the segments used for approximation, namely connected segments and disconnected segments. In connected segments, the ending point of the previous segment is the starting point of the new segment.…”

“…A schema for linear segments is presented in [64], consisting of a table, referred to as FunctionTable, where each row represents a linear model with attributes start time, end time, slope and intercept (i.e., base) of the segment. In case of connected segments [20], the end time attribute can be omitted.…”

“…When a new data tuple cannot be approximated by this line with the specified error bound, a new segment is started. In [20], two new piecewise linear approximation models were proposed, namely Swing and Slide, that achieve much higher compression compared to the cache and linear filters. We briefly discuss the swing and slide filters below.…”

“…( 1.19) Similar to the Fourier transform, by using the orthonormal basis functions ψ j,k , we can uniquely express a function f ∈ L 2 (R) as a linear combination of the basis functions ψ j,k as follows: 20) where < f, g >:= R f gdx is the usual inner product of two functions in L 2 (R).…”

A time-series compression technique and its application to the smart grid

“…Although time-series data and lossless compression have been investigated for quite some time, there is little research on lossy compression and retrieval of such data with guarantees. Studies such as [9,22,31] do not compress all types of data well or have disadvantages in terms of runtime. Moreover, none of these articles propose methods estimating the performance of their compression techniques.…”