P4cep

Abstract

In-network computing using programmable networking hardware is a strong trend in networking that promises to reduce latency and consumption of server resources through offloading to network elements (programmable switches and smart NICs). In particular, the data plane programming language P4 together with powerful P4 networking hardware has spawned projects offloading services into the network, e.g., consensus services or caching services. In this paper, we present a novel case for in-network computing, namely, Complex Event Processing (CEP). CEP processes streams of basic events, e.g., stemming from networked sensors, into meaningful complex events. Traditionally, CEP processing has been performed on servers or overlay networks. However, we argue in this paper that CEP is a good candidate for in-network computing along the communication path avoiding detouring streams to distant servers to minimize communication latency while also exploiting processing capabilities of novel networking hardware. We show that it is feasible to express CEP operations in P4 and also present a tool to compile CEP operations, formulated in our P4CEP rule specification language, to P4 code. Moreover, we identify challenges and problems that we have encountered to show future research directions for implementing full-fledged in-network CEP systems. CCS CONCEPTS• Networks → Programmable networks; In-network processing; • Software and its engineering → Publish-subscribe / event-based architectures; Message oriented middleware;

A Comprehensive Survey on Parallelization and Elasticity in Stream Processing

“…We see an emerging eld of new programmable network devices that allow for o oading computing from CPUs to the network, along with network programming languages such as P4. Early work on in-network CEP points to tremendous potential of in-network computing for this domain, but also reveals challenges due to limitations both in the programmable hardware as well as in the network programming language [77].…”

INDIANA—In-Network Distributed Infrastructure for Advanced Network Applications

“…We investigate the execution of stateful dataflow operators within the in-network computing space first. Due to limitations on switch-ASIC resources, several designs propose approximation (Jepsen et al, 2018), careful estimation (Gupta et al, 2018), or finite state machines (Kohler et al, 2018) for evaluating stateful operators. In addition, the absence of general-purpose computing elements such as loops, floating point, and arithmetic operators support also hinders the ability to use fast switching hardware.…”

“…Specifically, programmable networking components such as programmable Smart NICs and high-speed switching ASICs, processors and FPGAs are recent additions to the data processing ecosystem as strategies to introduce new packet processing protocols and management with ease. In addition to packet processing, these programmable components are now considered an attractive alternative to running specific network services for use cases such as consensus, packet classification, and deploying microservices (Choi et al, 2020; Dang et al, 2020; Ibanez et al, 2021; Kohler et al, 2018). With significant data processing capabilities, these components can lay the ground for achieving latency requirements for interactive applications.…”

“…Building a framework for deploying microservices within the networking domain comes with challenges, including a limited set of resources, sharing resources to facilitate existing networking functionality, and limited infrastructure access. Consequently, most application designers are unaware of how to utilize this high-speed infrastructure and benefit from low-latency application execution, limiting performance improvements to only those experts capable of designing custom solutions (Gupta et al, 2018; Jepsen et al, 2018; Kohler et al, 2018; Xiong and Zilberman, 2019; Zheng and Zilberman, 2021). The microservice framework also needs to support 1) service isolation, 2) elasticity, 3) stream and service scheduling, and 4) dynamic service deployment.…”

Programmable Event Detection for In-Band Network Telemetry

“…On the contrary, micro bursts may not be important for a service flow if end-to-end latency is lower than for example 3ms. Although complex event processing in the data plane on application layer data has been previously explored [20], [21], this paper applies it to network telemetry data.…”