Data stream processing with concurrency control

Abstract

A recent trend in data stream processing shows the use of advanced continuous queries (CQs) that reference non-streaming resources such as relational data in databases and machine learning models. Since non-streaming resources could be shared among multiple systems, resources may be updated by the systems during the CQ-execution. As a consequence, CQs may reference resources inconsistently, and lead to a wide range of problems from inappropriate results to fatal system failures. In this paper, we address this inconsistency problem by introducing the concept of transaction processing onto data stream processing.

DAGGTAX: A Taxonomy of Data Aggregation Processes

“…Oyamada et al [27] point out that the aggregation in a DSMS may also involve non-streaming data, which can be shared and updated by other processes, causing potential data inconsistency. The authors propose a concurrency control mechanism to prevent the inconsistency.…”

“…The retrieval of persistent raw data involves locating the data in the storage and the necessary I/O. Shared: Raw data of some DAP examples in Section 4 are read or updated by other processes at the same time when they are read for aggregation [3], [26], [27]. The same raw data may be aggregated by several DAPs, or accessed by processes that do not perform aggregations.…”

A Topic-based Dynamic Clustering Algorithm for Text Stream

“…There are three kinds of models to process the data stream, such as the time-limited model, the sliding window model and the snapshot model [3,4]. The data scale of all these three models depend on the selection of time interval, which are defined by the time interval from an initial time to current time, a certain time widow size and a certain time interval between each snapshot operation, respectively.…”