Scalable Online First-Order Monitoring

Abstract

Online monitoring is the task of identifying complex temporal patterns while incrementally processing streams of data-carrying events. Existing state-of-the-art monitors for first-order patterns, which may refer to and quantify over data values, can process streams of modest velocity in real-time. We show how to scale up first-order monitoring to substantially higher velocities by slicing the stream, based on the events' data values, into substreams that can be monitored independently. Because monitoring is not embarrassingly parallel in general, slicing can lead to data duplication. To reduce this overhead, we adapt hash-based partitioning techniques from databases to the monitoring setting. We implement these techniques in an automatic data slicer based on Apache Flink and empirically evaluate its performance using two tools-MonPoly and DejaVu-to monitor the substreams. Our evaluation attests to substantial scalability improvements for both tools.

Edge-Based Runtime Verification for the Internet of Things

“…We identify integration of such further techniques as avenues of possible future work. Advanced approaches can target scalability by performing slicing upon the event stream, by identifying substreams that can be independently monitored, or by exploiting hash-based partitioning techniques from databases research [32]. We note that our opportunistic, lightweight method of dealing with events excludes data repair techniques [33] in order to reduce complexity and load.…”

“…Furthermore, our work can be considered complementary to those that study different RV techniques and languages for IoT systems: with our greybox design, we can support different logics and verification tools with minimal impact on the service architecture itself. Various approaches and RV tools exist in the literature [45], [46], [47]; as we show in Sec. 4, in order to support them, it would suffice to replace the MFOTL verification engine and provide the appropriate application model.…”