A rule-based tool for gradual granular data aggregation

Abstract

In order to keep more detailed data available for longer periods, old data has to be reduced gradually to save space and improve query performance, especially on resource-constrained systems with limited storage and query processing capabilities. In this regard, some hand-coded data aggregation solutions have been developed; however, their actual usage have been limited, for the reason that hand-coded data aggregation solutions have proven themselves too complex to maintain. Maintenance need to occur as requirements change frequently and the existing data aggregation techniques lack flexibility with regards to efficient requirements change management. This paper presents an effective rule-based tool for data reduction based on gradual granular data aggregation. With the proposed solution, data can be maintained at different levels of granularity. The solution is based on high-level data aggregation rules. Based on these rules, data aggregation code can be auto-generated. The solution is effective, easy-to-use and easy-to-maintain. In addition, the paper also demonstrates the use of the proposed tool based on a farming case study using standard database technologies. The results show productivity of the proposed tool-based solution in terms of initial development time, maintenance time and alteration time as compared to a hand-coded solution.

Reducing Multidimensional Data

“…We also intend to extend our experiments by combining our own work on reductions with that concerning indexes in a multidimensional context [6]. At last we wish to apply the principles of reduction to a reel data sample of analytic domain such as banking or insurance etc.…”

“…The authors also propose techniques to query reduced multidimensional objects. As mentioned in [6], this work is highly theoretical but it fails to provide us a concrete example of implementation strategy. In [6], a gradual data aggregation solution based on conception, implementation and evaluation is proposed.…”

“…As mentioned in [6], this work is highly theoretical but it fails to provide us a concrete example of implementation strategy. In [6], a gradual data aggregation solution based on conception, implementation and evaluation is proposed. This solution is based on a table containing different temporal granularities: second, minute, hour, month and year.…”

“…[5] and [6] use a temporal table for gradual data reduction. Our goal is more ambitious as it aims to study data reduction of the complete multidimensional schema.…”

Organic streams: Data aggregation and integration based on individual needs

“…As for data aggregation [17][18][19][20][21][22], Ozdemir [17] has presented a reliable data aggregation and transmission protocol based on the concept of functional reputation, which can improve the reliability of data aggregation and transmission. Jung et al [18] proposed and developed two data aggregation mechanisms based on hybrid clustering: the combined clustering-based data aggregation mechanism and the adaptive clustering-based data aggregation, in order to improve both data aggregation and energy efficiency.…”