Tool for snapshotting of aggregated data from streaming data

Abstract

In the modern world, streaming data is widespread in a significant number of subject areas. At the same time, there is often a need for stream processing of data in real time. In stream processing, approximate algorithms, which have higher efficiency than exact algorithms, are in high demand, as well as stream state forecasting. In databases, materialized views are used to store query results, but most implementations do not have the ability to update them incrementally. Thus, there is a need in the market for a tool that builds incrementally updated materialized views of streaming data, and also makes it possible to forecast the state of a stream and use approximate algorithms for processing streaming data. In addition, due to the high diversity of streaming data, their sources and algorithms for their processing and forecasting, such a tool should be extensible. The author of the article has developed such a tool. In the article, the architecture and mechanism of functioning of the tool are reviewed. The prospects for its further development are also studied in the article.