Steam++ An Extensible End-to-end Framework for Developing IoT Data Processing Applications in the Fog

Abstract

IoT applications usually rely on cloud computing services to perform data analysis such as filtering, aggregation, classification, pattern detection, and prediction. When applied to specific domains, the IoT needs to deal with unique constraints. Besides the hostile environment such as vibration and electricmagnetic interference, resulting in malfunction, noise, and data loss, industrial plants often have Internet access restricted or unavailable, forcing us to design stand-alone fog and edge computing solutions. In this context, we present STEAM++, a lightweight and extensible framework for real-time data stream processing and decision-making in the network edge, targeting hardware-limited devices, besides proposing a micro-benchmark methodology for assessing embedded IoT applications. In real-case experiments in a semiconductor industry, we processed an entire data flow, from values sensing, processing and analysing data, detecting relevant events, and finally, publishing results to a dashboard. On average, the application consumed less than 500kb RAM and 1.0% of CPU usage, processing up to 239 data packets per second and reducing the output data size to 14% of the input raw data size when notifying events.

A lightweight parallel execution engine for IoT data analytics in fog computing

“…Maleki, Neda [5] proposed So-FA, a Spark-oriented Fog architecture that leverages Spark functionalities to provide higher system utilization, energy efficiency, and scalability. Gomes, M. M. , et al [6] proposed STEAM++, a lightweight programming model and framework to simplify the development of end-to-end IoT applications for real-time data stream processing and decision-making, targeting resource limited devices located at the network edge. R. Dautov, et al [7] proposed a novel distributed stream processing architecture, broadening the principle of enabling processing closer to data sources adopted by Fog Computing.…”