Piecewise linear approximation of streaming time series data with max-error guarantees

Luo, Ge; Yi, Ke; Cheng, Siu-Wing; Li, Zhenguo; Fan, Wei; He, Cheng; Mu, Yadong

doi:10.1109/icde.2015.7113282

Cited by 61 publications

(52 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To fully capture the information from water quality data, we employ similar feature extraction procedure by using the latest 12 hours water quality data and extracting statistical features (mean, variance, maximum, minimum, skewness and kurtosis), frequency related features (FFT [15] and DWT [16]) and time series features (autocorrelation, PAA [13], PLA [14]) for each of the time series in the historical water quality data (RC, turbidity, and pH).…”

Section: Water Qualitymentioning

confidence: 99%

Predicting Urban Water Quality with Ubiquitous Data - A Data-driven Approach

Liu

Liang

Ouyang

et al. 2020

IEEE Trans. Big Data

View full text Add to dashboard Cite

Urban water quality is of great importance to our daily lives. Prediction of urban water quality help control water pollution and protect human health. However, predicting the urban water quality is a challenging task since the water quality varies in urban spaces non-linearly and depends on multiple factors, such as meteorology, water usage patterns, and land uses. In this work, we forecast the water quality of a station over the next few hours from a data-driven perspective, using the water quality data and water hydraulic data reported by existing monitor stations and a variety of data sources we observed in the city, such as meteorology, pipe networks, structure of road networks, and point of interests (POIs). First, we identify the influential factors that affect the urban water quality via extensive experiments. Second, we present a multi-task multi-view learning method to fuse those multiple datasets from different domains into an unified learning model. We evaluate our method with real-world datasets, and the extensive experiments verify the advantages of our method over other baselines and demonstrate the effectiveness of our approach.

show abstract

Section: Water Qualitymentioning

confidence: 99%

Predicting Urban Water Quality with Ubiquitous Data - A Data-driven Approach

Liu

Liang

Ouyang

et al. 2020

IEEE Trans. Big Data

View full text Add to dashboard Cite

show abstract

“…Another PLA algorithm with a constant update time has been introduced in [ 12 ]. However, again their approach is buffer-based as well with a worst-case space complexity of

and uses in their experiments around 1 KB memory.…”

Section: Related Workmentioning

confidence: 99%

Time and Memory Efficient Online Piecewise Linear Approximation of Sensor Signals

Grützmacher

Beichler

Hein

et al. 2018

Sensors

View full text Add to dashboard Cite

Piecewise linear approximation of sensor signals is a well-known technique in the fields of Data Mining and Activity Recognition. In this context, several algorithms have been developed, some of them with the purpose to be performed on resource constrained microcontroller architectures of wireless sensor nodes. While microcontrollers are usually constrained in computational power and memory resources, all state-of-the-art piecewise linear approximation techniques either need to buffer sensor data or have an execution time depending on the segment’s length. In the paper at hand, we propose a novel piecewise linear approximation algorithm, with a constant computational complexity as well as a constant memory complexity. Our proposed algorithm’s worst-case execution time is one to three orders of magnitude smaller and its average execution time is three to seventy times smaller compared to the state-of-the-art Piecewise Linear Approximation (PLA) algorithms in our experiments. In our evaluations, we show that our algorithm is time and memory efficient without sacrificing the approximation quality compared to other state-of-the-art piecewise linear approximation techniques, while providing a maximum error guarantee per segment, a small parameter space of only one parameter, and a maximum latency of one sample period plus its worst-case execution time.

show abstract

“…Also we select models that can be fitted incrementally to efficiently fit the data points online. To ensure the user-provided error bound is guaranteed for each data point, only models providing an error bound based on the uniform error norm are considered [34]. Last, we select models with lossless and lossy compression, allowing ModelarDB to select the approach most appropriate for each subsequence.…”

Section: Implementation Of User-defined Modelsmentioning

confidence: 99%

“…Methods have been proposed for online construction of approximations with minimal error [25], or maximal segment length for better compression [34]. As the optimal model can change over time, methods using multiple mathematical models have been developed.…”

Section: Related Workmentioning

confidence: 99%

ModelarDB

2018

View full text Add to dashboard Cite

Industrial systems, e.g., wind turbines, generate big amounts of data from reliable sensors with high velocity. As it is unfeasible to store and query such big amounts of data, only simple aggregates are currently stored. However, aggregates remove fluctuations and outliers that can reveal underlying problems and limit the knowledge to be gained from historical data. As a remedy, we present the distributed Time Series Management System (TSMS) ModelarDB that uses models to store sensor data. We thus propose an online, adaptive multi-model compression algorithm that maintains data values within a user-defined error bound (possibly zero). We also propose (i) a database schema to store time series as models, (ii) methods to push-down predicates to a key-value store utilizing this schema, (iii) optimized methods to execute aggregate queries on models, (iv) a method to optimize execution of projections through static code-generation, and (v) dynamic extensibility that allows new models to be used without recompiling the TSMS. Further, we present a general modular distributed TSMS architecture and its implementation, ModelarDB, as a portable library, using Apache Spark for query processing and Apache Cassandra for storage. An experimental evaluation shows that, unlike current systems, Mode-larDB hits a sweet spot and offers fast ingestion, good compression, and fast, scalable online aggregate query processing at the same time. This is achieved by dynamically adapting to data sets using multiple models. The system degrades gracefully as more outliers occur and the actual errors are much lower than the bounds.

show abstract

Piecewise linear approximation of streaming time series data with max-error guarantees

Cited by 61 publications

References 17 publications

Predicting Urban Water Quality with Ubiquitous Data - A Data-driven Approach

Predicting Urban Water Quality with Ubiquitous Data - A Data-driven Approach

Time and Memory Efficient Online Piecewise Linear Approximation of Sensor Signals

ModelarDB

Contact Info

Product

Resources

About