Synopses for probabilistic data over large domains

Abstract

Many real world applications produce data with uncertainties drawn from measurements over a continuous domain space. Recent research in the area of probabilistic databases has mainly focused on managing and querying discrete data in which the domain is limited to a small number of values (i.e. on the order of 10). When the size of the domain increases, current methods fail due to their nature of explicitly storing each value/probability pair. Such methods are not capable of extending their use to continuous-valued attributes. In this paper, we provide a scalable, accurate, space efficient probabilistic data synopsis for uncertain attributes defined over a continuous domain. Our synopsis construction methods are all error-aware to ensure that our synopsis provides an accurate representation of the underlying data given a limited space budget. Additionally, we are able to provide approximate query results over the synopsis with error bounds.We provide an extensive experimental evaluation to show that our proposed methods improve upon the current state of the art in terms of construction time and query accuracy. In particular, our synopsis can be constructed in O(N 2 ) time (where N is the number of tuples in the database). We also demonstrate the ability of our synopsis to answer a variety of interesting queries on a real data set and show that our query error is reduced by up to an order of magnitude over the previous state-of-the-art method.

Online Computing Quantile Summaries Over Uncertain Data Streams

“…Single aggregate, however, can be used only for simple applications [4]. Thus, [17], [46] selected a small set of represented data by clustering the uncertain data as the summarization; clustering points reserved important information of the data, however, when they represented the whole datasets, the precision was low and the errors were not bounded; additionally, they were incapable of trivially inducing the distribution of the datasets. Obtaining the distribution of a dataset is a fundamental issue in traditional data management, such as query analysis or query optimization.…”

“…Reference [5] summarized uncertain data streams by computing essential aggregates, but their methods did not apply to continuous domain either. To compute a synopses over continuous-valued datasets with uncertainty, [17] presented methods to find a representative set by clustering the datasets. This synopses, however, were capable of neither representing streaming data nor trivially inducing the distribution of the data.…”