Towards Concolic Testing for Hybrid Systems

Xuan

Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis

et al. 2020

Self Cite

Cyber-physical systems (CPSs) in critical infrastructure face a pervasive threat from attackers, motivating research into a variety of countermeasures for securing them. Assessing the effectiveness of these countermeasures is challenging, however, as realistic benchmarks of attacks are difficult to manually construct, blindly testing is ineffective due to the enormous search spaces and resource requirements, and intelligent fuzzing approaches require impractical amounts of data and network access. In this work, we propose active fuzzing, an automatic approach for finding test suites of packetlevel CPS network attacks, targeting scenarios in which attackers can observe sensors and manipulate packets, but have no existing knowledge about the payload encodings. Our approach learns regression models for predicting sensor values that will result from sampled network packets, and uses these predictions to guide a search for payload manipulations (i.e. bit flips) most likely to drive the CPS into an unsafe state. Key to our solution is the use of online active learning, which iteratively updates the models by sampling payloads that are estimated to maximally improve them. We evaluate the efficacy of active fuzzing by implementing it for a water purification plant testbed, finding it can automatically discover a test suite of flow, pressure, and over/underflow attacks, all with substantially less time, data, and network access than the most comparable approach. Finally, we demonstrate that our prediction models can also be utilised as countermeasures themselves, implementing them as anomaly detectors and early warning systems.

Section: Related Workmentioning

confidence: 99%

Active fuzzing for testing and securing cyber-physical systems

Xuan

Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis

et al. 2020

Self Cite

“…Etigowni et al [60] define a CPS control solution for securing power grids, focusing on information flow analyses based on (potentially verifiable) policy logic and symbolic execution. Beyond these examples, if a CPS can be modelled as a hybrid system, there are several formal methods that can be applied to it, including model checking [61], [62], SMT solving [63], non-standard analysis [64], process calculi [65], concolic testing [66], and theorem proving [67]. Defining a formal model that accurately characterises enough of the physical process and its interactions with the PLCs is, however, the hardest part.…”

Section: Related Workmentioning

confidence: 99%

Learning-Guided Network Fuzzing for Testing Cyber-Physical System Defences

2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE)

Sun

et al. 2019

Self Cite

The threat of attack faced by cyber-physical systems (CPSs), especially when they play a critical role in automating public infrastructure, has motivated research into a wide variety of attack defence mechanisms. Assessing their effectiveness is challenging, however, as realistic sets of attacks to test them against are not always available. In this paper, we propose smart fuzzing, an automated, machine learning guided technique for systematically finding 'test suites' of CPS network attacks, without requiring any knowledge of the system's control programs or physical processes. Our approach uses predictive machine learning models and metaheuristic search algorithms to guide the fuzzing of actuators so as to drive the CPS into different unsafe physical states. We demonstrate the efficacy of smart fuzzing by implementing it for two real-world CPS testbeds-a water purification plant and a water distribution system-finding attacks that drive them into 27 different unsafe states involving water flow, pressure, and tank levels, including six that were not covered by an established attack benchmark. Finally, we use our approach to test the effectiveness of an invariant-based defence system for the water treatment plant, finding two attacks that were not detected by its physical invariant checks, highlighting a potential weakness that could be exploited in certain conditions.

“…With these, the CPS can be modelled as a hybrid system and a variety of established techniques can be applied (e.g. model checking [40], SMT solving [41], non-standard analysis [42], concolic testing [43], runtime model validation [44], or theorem proving [45,46]). With discretised models of the physical part, classical modelling and verification techniques can also be applied, e.g.…”

Section: Related Workmentioning

confidence: 99%

Learning from Mutants: Using Code Mutation to Learn and Monitor Invariants of a Cyber-Physical System

2018 IEEE Symposium on Security and Privacy (SP)

Sun

2018

Self Cite

113

Cyber-physical systems (CPS) consist of sensors, actuators, and controllers all communicating over a network; if any subset becomes compromised, an attacker could cause significant damage. With access to data logs and a model of the CPS, the physical effects of an attack could potentially be detected before any damage is done. Manually building a model that is accurate enough in practice, however, is extremely difficult. In this paper, we propose a novel approach for constructing models of CPS automatically, by applying supervised machine learning to data traces obtained after systematically seeding their software components with faults ("mutants"). We demonstrate the efficacy of this approach on the simulator of a real-world water purification plant, presenting a framework that automatically generates mutants, collects data traces, and learns an SVM-based model. Using cross-validation and statistical model checking, we show that the learnt model characterises an invariant physical property of the system. Furthermore, we demonstrate the usefulness of the invariant by subjecting the system to 55 network and codemodification attacks, and showing that it can detect 85% of them from the data logs generated at runtime.