Modeling Large Time Series for Efficient Approximate Query Processing

Perera, Kasun S.; Hahmann, Martin; Lehner, Wolfgang; Pedersen, Torben Bach; Thomsen, Christian

doi:10.1007/978-3-319-22324-7_16

Cited by 5 publications

(6 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similarly, only tsdb [30], Plato [38] and PhilDB [43] are intended for use as general TSMSs and have implementations complete enough for this task based on the description in their respective publications. The remaining systems serve only as realizations of new theoretical methods, FAQ [31] is used for evaluating a method for model-based AQP, RINSE [34] demonstrates the capabilities of the ADS+ index [35], [36], Chronos [39] illustrates the benefits of write patterns optimized for flash storage but does neither provide thread-safety nor protection against data corruption, the system by Perera et al [37] is incomplete, and both Pytms and RoundRobinson [41], [42] are simple implementations of a formalism. In summary, researchers focusing on development of general purpose TSMSs are at the moment focused on systems that solve the problem at scale through distributed computing, with centralized systems being relegated to being test platforms for new theoretical methods.…”

Section: Discussionmentioning

confidence: 99%

“…In addition, Plato provides an interface for users to implement additional models, making it possible for domain experts to use models optimized for a specific domain. For the research systems, FAQ [31], RINSE [34] and the system Perera by et al [37] all support AQP using models, while Pytms and RoundRobinson [41], [42] only provide user-defined aggregates. None of the four research systems implemented functionality for stream processing.…”

Section: Discussionmentioning

confidence: 99%

“…A centralized TSMS that utilizes models for AQP was proposed by Perera et al [37]. The system manages a set of models for each time series and the TSMS optimizer selects the most appropriate model for each query, falling back to the raw data points if necessary.…”

Section: Systemsmentioning

confidence: 99%

See 2 more Smart Citations

Time Series Management Systems: A Survey

Jensen

Pedersen

Thomsen

2017

IEEE Trans. Knowl. Data Eng.

Self Cite

137

View full text Add to dashboard Cite

The collection of time series data increases as more monitoring and automation are being deployed. These deployments range in scale from an Internet of things (IoT) device located in a household to enormous distributed Cyber-Physical Systems (CPSs) producing large volumes of data at high velocity. To store and analyze these vast amounts of data, specialized Time Series Management Systems (TSMSs) have been developed to overcome the limitations of general purpose Database Management Systems (DBMSs) for times series management. In this paper, we present a thorough analysis and classification of TSMSs developed through academic or industrial research and documented through publications. Our classification is organized into categories based on the architectures observed during our analysis. In addition, we provide an overview of each system with a focus on the motivational use case that drove the development of the system, the functionality for storage and querying of time series a system implements, the components the system is composed of, and the capabilities of each system with regard to Stream Processing and Approximate Query Processing (AQP). Last, we provide a summary of research directions proposed by other researchers in the field and present our vision for a next generation TSMS.Comment: 20 Pages, 15 Figures, 2 Tables, Accepted for publication in IEEE TKD

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Time Series Management Systems: A Survey

Jensen

Pedersen

Thomsen

2017

IEEE Trans. Knowl. Data Eng.

Self Cite

137

View full text Add to dashboard Cite

show abstract

“…al. [29] propose offline algorithms for finding similarities between time series aggregates, in an OLAP cube, similar aggregates, can then be materialized as a model or as a model and an offset to reduce the size of a materialized cube. A similar method for online data cubes were proposed by Shaikh et.…”

Section: Related Workmentioning

confidence: 99%

“…Compared to other OLAP systems [35,20,10,38,9,36,19,11], ModelarDBv2 executes multi-dimensional aggregate queries on models. Also, while the existing model-based approaches for OLAP [29,33] store both the raw data points and model, ModelarDBv1 stores only the highly compressed models. In summary, ModelarDBv2 provides state-of-the-art compression and query performance for dimensional time series by compressing correlated time series as one sequence of model and executing OLAP queries on models.…”

Section: Related Workmentioning

confidence: 99%

Scalable Model-Based Management of Correlated Dimensional Time Series in ModelarDB+

Jensen,

Pedersen,

Thomsen

2019

Preprint

Self Cite

View full text Add to dashboard Cite

To monitor critical infrastructure, high quality sensors sampled at a high frequency are increasingly installed. However, due to the big amounts of data produced, only simple aggregates are stored. This removes outliers and hides fluctuations that could indicate problems. As a solution we propose compressing time series with dimensions using a model-based method we name Multi-model Group Compression (MMGC). MMGC adaptively compresses groups of correlated time series with dimensions using an extensible set of models within a user-defined error bound (possibly zero). To partition time series into groups, we propose a set of primitives for efficiently describing correlation for data sets of varying sizes. We also propose efficient query processing algorithms for executing multi-dimensional aggregate queries on models instead of data points. Last, we provide an open-source implementation of our methods as extensions to the model-based Time Series Management System (TSMS) Mod-elarDB. ModelarDB interfaces with the stock versions of Apache Spark and Apache Cassandra and thus can reuse existing infrastructure. Through an evaluation we show that, compared to widely used systems, our extended ModelarDB provides up to 11 times faster ingestion due to high compression, 65 times better compression due to the adaptivity of MMGC, 92 times faster aggregate queries as they are executed on models, and close to linear scalability while also being extensible and supporting online query processing.

show abstract

A Fuzzy Querying Using Cooperative Answers and Proximity Measure

Aggoune

2019

Smart Innovation, Systems and Technologies

View full text Add to dashboard Cite

Modeling Large Time Series for Efficient Approximate Query Processing

Cited by 5 publications

References 7 publications

Time Series Management Systems: A Survey

Time Series Management Systems: A Survey

Scalable Model-Based Management of Correlated Dimensional Time Series in ModelarDB+

A Fuzzy Querying Using Cooperative Answers and Proximity Measure

Contact Info

Product

Resources

About