Using datacube aggregates for approximate querying and deviation detection

Abstract

Abstract-Much research has been devoted to the efficient computation of relational aggregations and, specifically, the efficient execution of the datacube operation. In this paper, we consider the inverse problem, that of deriving (approximately) the original data from the aggregates. We motivate this problem in the context of two specific application areas, approximate query answering and data analysis. We propose a framework based on the notion of information entropy that enables us to estimate the original values in a data set, given only aggregated information about it. We then show how approximate queries on the data from which the aggregates were derived can be performed using our framework. We also describe an alternate use of the proposed framework that enables us to identify values that deviate from the underlying data distribution, suitable for data mining purposes. We present a detailed performance study of the algorithms using both real and synthetic data, highlighting the benefits of our approach as well as the efficiency of the proposed solutions. Finally, we evaluate our techniques with a case study on a real data set, which illustrates the applicability of our approach.

Improving estimation accuracy of aggregate queries on data cubes

“…The Approximate Query Answering (AQUA) (Gibbons & Matias 1998) system provides approximate answers using small precomputed synopses of the underlying base data. In (Palpanas et al 2005), the authors consider the problem of deriving approximately the original data from the aggregates. They propose a framework for estimating the original values based on the notion of information entropy.…”

“…In such cases, it is quite useful to generate an estimate or approximate answers using approximate query processing techniques. A key issue is the accuracy of the estimates for aggregate queries (e.g., queries computing SUM or COUNT expressions), and was the focus of recent research activity (e.g., (Palpanas, Koudas & Mendelson 2005), (Pourabbas & Shoshani 2007)). …”

Measure-driven keyword-query expansion

“…The principle maintains that since we have no plausible reason to bias the distribution towards a certain form, the reconstructed distribution should be as uniform and "uninformative" as possible, subject to the observed constraints. The technique has been successfully applied for purposes similar to our own before [14,16,12,13].…”

“…, wr). The estimation approach that we use is based on the widely accepted Principle of Maximum Entropy [8] and has been successfully employed before [14,16,12,13].…”

Knowledge Mining for the Business Analyst

“…Nevertheless, the proposed solutions are specific to the particular tasks for which they were developed. Previous work has also studied the problem of applying data mining techniques in the context of data warehouses [14,16,17,13], which are used by business analysts during the decision-making process.…”