Sliding windows over uncertain data streams

Abstract

Uncertain data streams can have tuples with both value and existential uncertainty. A tuple has value uncertainty when it can assume multiple possible values. A tuple is existentially uncertain when the sum of the probabilities of its possible values is <1. A situation where existential uncertainty can arise is when applying relational operators to streams with value uncertainty. Several prior works have focused on querying and mining data streams with both value and existential uncertainty. However, none of them have studied, in depth, the implications of existential uncertainty on sliding window processing, even though it naturally arises when processing uncertain data. In this work, we study the challenges arising from existential uncertainty, more specifically the management of count-based sliding windows, which are a basic building block of stream processing applications. We extend the semantics of sliding window to define the novel concept of uncertain sliding windows and provide both exact and approximate algorithms for managing windows under existential uncertainty. We also show how current state-of-the-art techniques for answering similarity join queries can be easily adapted to be used with uncertain sliding windows. We evaluate our proposed techniques under a variety of configurations using real data. The results show that the algorithms used to maintain uncertain sliding windows can efficiently operate while providing a highquality approximation in query answering. In addition, we show that sort-based similarity join algorithms can perform better than index-based techniques (on 17 real datasets) when the number of possible values per tuple is low, as in many real-world applications.

Predictive intelligence to the edge: impact on edge analytics

“…As an example, an analytics query over  could be: 'continuously return all context vectors of the past hour, i.e., N=60 min'. The sliding window is the most widely used in continuous aggregation and fusion analytics functions (Dallachiesa et al 2015;Patroumpas and Sellis 2011;Abadi et al 2003Abadi et al , 2005.…”

Using community information for natural disaster alerts

“…Therefore, how much information should be used for analysis is an important question. Many studies have focused on the improvement of efficiency to obtain good performance in time-series data mining [19], that is, by employing sliding window [20–22] and deep learning [13,23] approaches. Deep learning models can take into account the characteristics of time-series data for prediction [23], but training deep networks is usually a very time-consuming task [24].…”

Conditional MaxRS Query for Evolving Spatial Data

“…The issue of real-time query processing and indexing over spatio-temporal streaming data have been addressed extensively in prior literature, e.g., Hart et al, 2005 ; Mokbel et al, 2005 ; Dallachiesa et al, 2015 , etc. For real-time computation, it is necessary to restrict the set of inspected data points at any time using techniques such as punctuation (embedded annotations), synopses (data summaries), windows (e.g., sliding windows—only items received in past t minutes), etc.…”

“…The proposed system utilized the idea of predicate-based sliding windows, and employed an incremental evaluation paradigm by continuously updating the query answer over a window. Dallachiesa et al ( 2015 ) proposed both exact and approximate algorithms to manage count-based uncertain sliding windows for uncertain data streams (e.g., tuples can have both value and existential uncertainty). In contrast to these traditional window-based settings, we process C-MaxRS query in an event-based manner using all the data points received so far.…”