Multi-dimensional Probabilistic Regression over Imprecise Data Streams

Abstract

In applications of Web of Things or Web of Events, a massive volume of multi-dimensional streaming data are automatically and continuously generated from different sources, such as GPS, sensors, and other measurement devices, which are essentially imprecise (inaccurate and/or uncertain). It is challenging to monitor and get insights over imprecise and low-level streaming data, in order to capture potentially important data changing trends and to initiate prompt responses. In this work, we investigate solutions for conducting multi-dimensional and multi-granularity probabilistic regression for the imprecise streaming data. The probabilistic nature of streaming data poses big computational challenges to the regression and its aggregation. In this paper, we study a series of techniques on multi-dimensional probabilistic regression, including aggregation, sketching, popular path materialization, and exception-driven querying. Extensive experiments on real and synthetic datasets demonstrate the efficiency and scalability of our proposals.

Meta-Sketch: A Neural Data Structure for Estimating Item Frequencies of Data Streams

“…Basic sketches, including CM-sketch (Cormode and Muthukrishnan 2005), C-sketch (Charikar, Chen, and Farach-Colton 2002) and CU-sketch (Estan and Varghese 2002), use a 2D array of counters as the core structure. Some varients (Li et al 2020;Zhong et al 2021;Gao et al 2022;Liu and Xie 2021) broaden application scenarios based on basic sketches. To optimize the sketching performance, several augmented sketches (Roy, Khan, and Alonso 2016;Zhou et al 2018) were proposed, which attach filters to basic sketches, to capture the preliminary patterns of skewed distributions (e.g., high/low-frequency items).…”