Model Conformance for Cyber-Physical Systems

Roehm, Hendrik; Oehlerking, Jens; Woehrle, Matthias; Althoff, Matthias

doi:10.1145/3306157

Cited by 43 publications

(35 citation statements)

References 130 publications

(178 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…We did not find redundant mutants either, which was expected since (1) mutations concerning different data items, by definition, are expected to lead to different outputs, (2) the set of operators applied to a same data item, if selected according to our methodology, cannot lead to mutated data that is redundant. When analysing pairs of seemingly redundant mutants, we identified five reasons why these mutants were actually not redundant: (1) the test case does not distinguish failures across data items (e.g., temperature values collected by different sensors), (2) the test case does not distinguish errors across different messages (e.g., in ESAT1-ADCS, the IfHK message reporting a broken sensor or the message sent by a sensor reporting malfunction), (3) the test case does not distinguish between errors in nominal and non-nominal data (e.g., it does not distinguish between VOR and FVOR), (4) the test case does not distinguish between upper and lower bounds (e.g., the mutants for VOR lead to the same assertion failures), and (5) the test case does not distinguish between different error codes (i.e, it simply verifies that an error code is generated).…”

Section: Resultsmentioning

confidence: 76%

“…More specifi-cally, the benchmark includes (1) the on-board embedded software system for ESAT1 [17], a maritime microsatellite recently launched into space, and (2) a configuration library used in constellations of nanosatellites [18]. Our empirical results show that DaMAT (1) successfully identifies different types of shortcomings in test suites, (2) prevents the introduction of equivalent and redundant mutants, and (3) is practically applicable in the CPS context.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Data-driven Mutation Analysis for Cyber-Physical Systems

Viganò¹,

Cornejo²,

Pastore³

et al. 2022

Preprint

View full text Add to dashboard Cite

Cyber-physical systems (CPSs) typically consist of a wide set of integrated, heterogeneous components; consequently, most of their critical failures relate to the interoperability of such components. Unfortunately, most CPS test automation techniques are preliminary and industry still heavily relies on manual testing. With potentially incomplete, manually-generated test suites, it is of paramount importance to assess their quality. Though mutation analysis has demonstrated to be an effective means to assess test suite quality in some specific contexts, we lack approaches for CPSs. Indeed, existing approaches do not target interoperability problems and cannot be executed in the presence of black-box or simulated components, a typical situation with CPSs. In this paper, we introduce data-driven mutation analysis, an approach that consists in assessing test suite quality by verifying if it detects interoperability faults simulated by mutating the data exchanged by software components. To this end, we describe a data-driven mutation analysis technique (DaMAT ) that automatically alters the data exchanged through data buffers. Our technique is driven by fault models in tabular form where engineers specify how to mutate data items by selecting and configuring a set of mutation operators. We have evaluated DaMAT with CPSs in the space domain; specifically, the test suites for the software systems of a microsatellite and nanosatellites launched on orbit last year. Our results show that the approach effectively detects test suite shortcomings, is not affected by equivalent and redundant mutants, and entails acceptable costs.

show abstract

Section: Resultsmentioning

confidence: 76%

Section: Introductionmentioning

confidence: 99%

Data-driven Mutation Analysis for Cyber-Physical Systems

Viganò¹,

Cornejo²,

Pastore³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…This requires non-deterministic models with potentially uncertain inputs, initial states, and parameters. A special form of conformance checking-called reachset conformance checking-ensures that all recorded behaviors of a real system are captured by the abstract non-deterministic model [24,25].…”

Section: Difficulty and Usefulnessmentioning

confidence: 99%

Set Propagation Techniques for Reachability Analysis

Althoff

Frehse

Girard

2021

Annu. Rev. Control Robot. Auton. Syst.

Self Cite

170

View full text Add to dashboard Cite

Reachability analysis consists in computing the set of states that are reachable by a dynamical system from all initial states and for all admissible inputs and parameters. It is a fundamental problem motivated by many applications in formal verification, controller synthesis, and estimation, to name only a few. This article focuses on a class of methods for computing a guaranteed overapproximation of the reachable set of continuous and hybrid systems, relying predominantly on set propagation; starting from the set of initial states, these techniques iteratively propagate a sequence of sets according to the system dynamics. After a review of set representation and computation, the article presents the state of the art of set propagation techniques for reachability analysis of linear, nonlinear, and hybrid systems. It ends with a discussion of successful applications of reachability analysis to real-world problems. Expected final online publication date for the Annual Review of Control, Robotics, and Autonomous Systems, Volume 4 is May 3, 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

show abstract

“…Related work has shown different approaches to perform such validations. Examples are conformance tests [23,34], unit tests [11,21], or distributed assertions [24,27].…”

Section: Usementioning

confidence: 99%

Non-Intrusive Distributed Tracing of Wireless IoT Devices with the FlockLab 2 Testbed

Trüb

Forno

Daschinger

et al. 2021

ACM Trans. Internet Things

View full text Add to dashboard Cite

Testbeds for wireless IoT devices facilitate testing and validation of distributed target nodes. A testbed usually provides methods to control, observe, and log the execution of the software. However, most of the methods used for tracing the execution require code instrumentation and change essential properties of the observed system. Methods that are non-intrusive are typically not applicable in a distributed fashion due to a lack of time synchronization or necessary hardware/software support. In this article, we present a tracing system for validating time-critical software running on multiple distributed wireless devices that does not require code instrumentation, is non-intrusive and is designed to trace the distributed state of an entire network. For this purpose, we make use of the on-chip debug and trace hardware that is part of most modern microcontrollers. We introduce a testbed architecture as well as models and methods that accurately synchronize the timestamps of observations collected by distributed observers. In a case study, we demonstrate how the tracing system can be applied to observe the distributed state of a flooding-based low-power communication protocol for wireless sensor networks. The presented non-intrusive tracing system is implemented as a service of the publicly accessible open source FlockLab 2 testbed.

show abstract

Model Conformance for Cyber-Physical Systems

Cited by 43 publications

References 130 publications

Data-driven Mutation Analysis for Cyber-Physical Systems

Data-driven Mutation Analysis for Cyber-Physical Systems

Set Propagation Techniques for Reachability Analysis

Non-Intrusive Distributed Tracing of Wireless IoT Devices with the FlockLab 2 Testbed

Contact Info

Product

Resources

About