Simulation and Real-World Evaluation of Attack Detection Schemes

Abstract

A variety of anomaly detection schemes have been proposed to detect malicious attacks to Cyber-Physical Systems. Among these schemes, Dynamic Watermarking methods have been proven highly effective at detecting a wide range of attacks. Unfortunately, in contrast to other anomaly detectors, no method has been presented to design a Dynamic Watermarking detector to achieve a user-specified false alarm rate, or subsequently evaluate the capabilities of an attacker under such a selection. This paper describes methods to measure the capability of an attacker, to numerically approximate this metric, and to design a Dynamic Watermarking detector that can achieve a user-specified rate of false alarms. The performance of the Dynamic Watermarking detector is compared to three classical anomaly detectors in simulation and on a real-world platform. These experiments illustrate that the attack capability under the Dynamic Watermarking detector is comparable to those of classic anomaly detectors. Importantly, these experiments also make clear that the Dynamic Watermarking detector is consistently able to detect attacks that the other class of detectors are unable to identify.

A Dynamic Watermarking Algorithm for Finite Markov Decision Problems

“…SUch model was further generalized in [31] as an LTI system with partial state observations, and in [32] as a networked control system. Using dynamic watermarking to detect against attacks on sensors for real-world systems, such as robotics mobility platform [33], multi-vehicle transportation system testbed [34], and automatic power generation system [35] are also reported respectively.…”

Detecting Deception Attacks on Autonomous Vehicles via Linear Time-Varying Dynamic Watermarking

“…This paper focuses on a particular detection method, Dynamic Watermarking, which adds a watermark signal to the control input to secure the measurement signals [4], [14]- [18]. Dynamic Watermarking is an active method meaning it makes alterations to the system which allow it to detect more complex attacks [19]- [21]. While, the specific methods of Dynamic Watermarking vary, in one popular approach the watermark is a multivariate Gaussian sequence that is generated by the controller.…”

Experimental evaluation of sensor attacks and defense mechanisms in feedback systems

“…Although the controller with an integral action leads to a larger attack impact than a controller without an integral action both in our work as well as in (Urbina et al, 2016), we show that the integral action prevents the attack from being stealthy. Porter et al (2019) compare four different detectors with respect to their attack detection capability in simulation and real-world experiments. In the present work, we will only compare two of the four detectors in-vestigated in (Porter et al, 2019), but look at a more sophisticated attack strategy.…”

“…Porter et al (2019) compare four different detectors with respect to their attack detection capability in simulation and real-world experiments. In the present work, we will only compare two of the four detectors in-vestigated in (Porter et al, 2019), but look at a more sophisticated attack strategy. We also use a technique similar to the watermarking in (Porter et al, 2019) to achieve a detection of stealthy attacks.…”

“…In the present work, we will only compare two of the four detectors in-vestigated in (Porter et al, 2019), but look at a more sophisticated attack strategy. We also use a technique similar to the watermarking in (Porter et al, 2019) to achieve a detection of stealthy attacks. However, instead of injecting an additive noise signal to the controller output, u(k), we inject the noise directly into the controller dynamics in this work.…”