Monitoring with Verified Guarantees

Abstract

Runtime monitoring is generally considered a light-weight alternative to formal verification. In safety-critical systems, however, the monitor itself is a critical component. For example, if the monitor is responsible for initiating emergency protocols, as proposed in a recent aviation standard, then the safety of the entire system critically depends on guarantees of the correctness of the monitor. In this paper, we present a verification extension to the Lola monitoring language that integrates the efficient specification of the monitor with Hoare-style annotations that guarantee the correctness of the monitor specification. We add two new operators, assume and assert, which specify assumptions of the monitor and expectations on its output, respectively. The validity of the annotations is established by an integrated SMT solver. We report on experience in applying the approach to specifications from the avionics domain, where the annotation with assumptions and assertions has lead to the discovery of safety-critical errors in the specifications. The errors range from incorrect default values in offset computations to complex algorithmic errors that result in unexpected temporal patterns.

VeriMon: A Formally Verified Monitoring Tool

“…Several of VeriMon's features, such as the non-recursive let operator and the improved algorithms for Since and Until, have been propagated back to MonPoly and have guided the design of a new monitoring tool implemented in C ++ , CPPMon. 7 We are happy to start seeing other work in the community that uses proof assistants [4][5][6]17], deductive verifiers [9], or SMT solvers [7,14] to improve the trustworthiness of monitors. We believe that formal verification is the only way towards a landscape of tools that are reliable and maintainable: not just one-paper wonders.…”

A Hierarchy of Monitoring Properties for Autonomous Systems

“…Given a RTLola specification, the framework performs various static analyses to provide reliable guarantees prior to execution, e.g., to determine a memory bound or verify stream behaviors [22]. It then either directly executes the specification with the RTLola-Interpreter [23] or compiles it to software [24] or hardware [25].…”

“…For instance, RTLola is designed to analyze the memory to give bounds on the memory consumption of the generated monitor implementation. Further, specification analysis can help to check consistency and to provide verified guarantees on the behavior of the monitor [22] that help argue ASTM monitor coverage. Automatic generation of monitor implementation.…”

Explainable Online Monitoring of Metric Temporal Logic

“…Some verified monitors were developed recently using proof assistants, e.g., VeriMon [29] and Vydra [28] in Isabelle and lattice-mtl [8] in Coq. Others leveraged SMT technology to increase their trustworthiness [12,14]. To the best of our knowledge, we present the first verified checker for an online monitor's output, even though verified certifiers are standard practice in other areas such as distributed systems [35], model checking [37,38], and SAT solving [11,21].…”