Continuously monitoring top-k uncertain data streams: a probabilistic threshold method

Hua, Ming; Pei, Jian

doi:10.1007/s10619-009-7043-x

Cited by 11 publications

(6 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The aim is to identify those objects that have the possibility to be among the top-k objects. In [33], a probabilistic threshold-based method for dealing with queries over uncertain data streams is proposed. The following algorithms are presented: (a) a deterministic exact algorithm, (b) a randomized method, and (c) their space-efficient versions using quantile summaries.…”

Section: Ordered Sets Queries Managementmentioning

confidence: 99%

An intelligent, uncertainty driven aggregation scheme for streams of ordered sets

Kolomvatsos

2016

Appl Intell

View full text Add to dashboard Cite

Data streams management has attracted the attention of many researchers during the recent years. The reason is that numerous devices generate huge amounts of data demanding an efficient processing scheme for delivering high quality applications. Data are reported through streams and stored into a number of partitions. Separation techniques facilitate the parallel management of data while intelligent methods are necessary to manage these multiple instances of data. Progressive analytics over huge amounts of data could be adopted to deliver partial responses and, possibly, to save time in the execution of applications. An interesting research domain is the efficient management of queries over multiple partitions. Usually, such queries demand responses in the form of ordered sets of objects (e.g., top-k queries). These ordered sets include objects in a ranked order and require novel mechanisms for deriving responses based on partial results. In this paper, we study a setting of multiple data partitions and propose an intelligent, uncertainty driven decision making mechanism that aims to respond to streams of queries. Our mechanism delivers an ordered set of objects over a number of partial ordered subsets retrieved by each partition of data. We envision that a number of query processors are placed in front of each partition and report progressive analytics to a Query Controller (QC). The QC receives queries, assigns the task to the underlying processors and decides the right time to deliver the final ordered set to the application. We propose an aggregation model for deriving the final ordered set of objects and a Fuzzy Logic (FL) inference process. We present a Type-2 FL system that decides when the QC should stop aggregating partial subsets and return the final response to the application. We report on the performance of the proposed mechanism through the execution of a large set of experiments. Our results deal with the throughput of the QC, the quality of the final ordered set of objects and the time required for delivering the final response.

show abstract

Section: Ordered Sets Queries Managementmentioning

confidence: 99%

An intelligent, uncertainty driven aggregation scheme for streams of ordered sets

Kolomvatsos

2016

Appl Intell

View full text Add to dashboard Cite

show abstract

“…Hence, we can approximate the optimal β 1 value for (8) by finding the β 1 value that minimizes the RHS of (11). Let f (β 1 ) be the RHS of (11) and take its derivative w.r.t.…”

Section: A Improved Bounds On Pr[y > γ]mentioning

confidence: 99%

“…Monitoring centralized uncertain data for top-k and similarity queries were studied in [11], [17], [35]. On the other hand, due to their importance and numerous applications, constraint and function monitoring with thresholds on deterministic distributed data were examined extensively, e.g., [4], [12], [16], [19], [23], [29].…”

Section: Related Workmentioning

confidence: 99%

Efficient Threshold Monitoring for Distributed Probabilistic Data

Tang

Phillips

et al. 2012

2012 IEEE 28th International Conference on Data Engineering

View full text Add to dashboard Cite

Abstract-In distributed data management, a primary concern is monitoring the distributed data and generating an alarm when a user specified constraint is violated. A particular useful instance is the threshold based constraint, which is commonly known as the distributed threshold monitoring problem . This work extends this useful and fundamental study to distributed probabilistic data that emerge in a lot of applications, where uncertainty naturally exists when massive amounts of data are produced at multiple sources in distributed, networked locations. Examples include distributed observing stations, large sensor fields, geographically separate scientific institutes/units and many more. When dealing with probabilistic data, there are two thresholds involved, the score and the probability thresholds. One must monitor both simultaneously, as such, techniques developed for deterministic data are no longer directly applicable. This work presents a comprehensive study to this problem. Our algorithms have significantly outperformed the baseline method in terms of both the communication cost (number of messages and bytes) and the running time, as shown by an extensive experimental evaluation using several, real large datasets.

show abstract

“…Hua and Pay [17] also examined the continuous evaluation of top-k queries over uncertain data streams. They proposed a novel uncertain data stream model and introduced the continuous probabilistic threshold top-k queries.…”

Section: Det and Sammentioning

confidence: 99%

Continuous Processing of Preference Queries in Data Streams

Kontaki

Papadopoulos

Manolopoulos

2010

SOFSEM 2010: Theory and Practice of Computer Science

View full text Add to dashboard Cite

Abstract. Preference queries have received considerable attention in the recent past, due to their use in selecting the most preferred objects, especially when the selection criteria are contradictory. Nowadays, a significant number of applications require the manipulation of time evolving data and therefore the study of continuous query processing has recently attracted the interest of the data management community. The goal of continuous query processing is to continuously evaluate longrunning queries by using incremental algorithms and thus to avoid query evaluation from scratch, if possible. In this paper, we examine the characteristics of important preference queries, such as skyline, top-k and top-k dominating and we review algorithms proposed for the evaluation of continuous preference queries under the sliding window streaming model.

show abstract

Continuously monitoring top-k uncertain data streams: a probabilistic threshold method

Cited by 11 publications

References 56 publications

An intelligent, uncertainty driven aggregation scheme for streams of ordered sets

An intelligent, uncertainty driven aggregation scheme for streams of ordered sets

Efficient Threshold Monitoring for Distributed Probabilistic Data

Continuous Processing of Preference Queries in Data Streams

Contact Info

Product

Resources

About