Algorithms for Windowed Aggregations and Joins on Distributed Stream Processing Systems

Verwiebe, Juliane; Grulich, Philipp M.; Traub, Jonas; Markl, Volker

doi:10.1007/s13222-022-00417-y

Cited by 3 publications

(1 citation statement)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…NebulaStream aims to provide unique aggregation capabilities, innovates and integrates various technologies, including cloud, fog and sensor networks, to create a unified sensor-fog-cloud environment and facilitate the development of foreseeable IoT applications. Furthermore, the system follows the design principle of maximized sharing [84]. NebulaStream addresses the challenges of heterogeneity, unreliability and scalability of EC-IoT by unifying the end, network and cloud edge infrastructure to enable real-time processing of data and efficient utilization of underlying heterogeneous and geo-distributed resources.…”

Section: • Nebulastreammentioning

confidence: 99%

Application and Research of IoT Architecture for End-Net-Cloud Edge Computing

Zhang

Zhou

et al. 2022

Electronics

View full text Add to dashboard Cite

At the edge of the network close to the source of the data, edge computing deploys computing, storage and other capabilities to provide intelligent services in close proximity and offers low bandwidth consumption, low latency and high security. It satisfies the requirements of transmission bandwidth, real-time and security for Internet of Things (IoT) application scenarios. Based on the IoT architecture, an IoT edge computing (EC-IoT) reference architecture is proposed, which contained three layers: The end edge, the network edge and the cloud edge. Furthermore, the key technologies of the application of artificial intelligence (AI) technology in the EC-IoT reference architecture is analyzed. Platforms for different EC-IoT reference architecture edge locations are classified by comparing IoT edge computing platforms. On the basis of EC-IoT reference architecture, an industrial Internet of Things (IIoT) edge computing solution, an Internet of Vehicles (IoV) edge computing architecture and a reference architecture of the IoT edge gateway-based smart home are proposed. Finally, the trends and challenges of EC-IoT are examined, and the EC-IoT architecture will have very promising applications.

show abstract

Section: • Nebulastreammentioning

confidence: 99%

Application and Research of IoT Architecture for End-Net-Cloud Edge Computing

Zhang

Zhou

et al. 2022

Electronics

View full text Add to dashboard Cite

show abstract

Evolving cybersecurity frontiers: A comprehensive survey on concept drift and feature dynamics aware machine and deep learning in intrusion detection systems

Shyaa,

Ibrahim,

Zainol

et al. 2024

Engineering Applications of Artificial Intelligence

View full text Add to dashboard Cite

Low-Latency Adaptive Distributed Stream Join System Based on a Flexible Join Model

Wang,

Zuo,

Zhang

et al. 2024

Proc. ACM Manag. Data

View full text Add to dashboard Cite

Stream join is a fundamental operation in stream processing and has attracted extensive research due to its large resource consumption and serious impact on system performance. As the theoretical basis of stream join systems, the stream join model greatly affects system performance. State-of-the-art stream join models either consume too much computing resources or too much storage resources, thus resulting in lower throughput or higher latency. In this paper, we propose a new stream join model for processing arbitrary join predicates, called CoModel, which offers a flexible trade-off between memory and computing resource consumption. More importantly, CoModel can achieve the minimum sum of the number of store operations and join operations among all existing join models, and thus can achieve the lowest latency and highest throughput when the overheads associated with the local stream join for each input tuple are approximately constant. We give a trade-off strategy for CoModel and theoretically prove its performance advantages based on queuing theory. Furthermore, we design and implement an adaptive distributed stream join system, CoStream, based on CoModel. CoStream can adaptively adjust its structure according to resource constraints and statistics of input data. We conduct extensive experiments for CoStream to evaluate its performance and adaptivity, and the results show that CoStream has the lowest latency and highest throughput in various scenarios.

show abstract

Algorithms for Windowed Aggregations and Joins on Distributed Stream Processing Systems

Cited by 3 publications

References 27 publications

Application and Research of IoT Architecture for End-Net-Cloud Edge Computing

Application and Research of IoT Architecture for End-Net-Cloud Edge Computing

Evolving cybersecurity frontiers: A comprehensive survey on concept drift and feature dynamics aware machine and deep learning in intrusion detection systems

Low-Latency Adaptive Distributed Stream Join System Based on a Flexible Join Model

Contact Info

Product

Resources

About