Causality-Aided Falsification

Akazaki, Takumi; Kumazawa, Yoshihiro; Hasuo, Ichiro

doi:10.4204/eptcs.257.2

Cited by 8 publications

(14 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…So far, the falsification problem has received extensive industrial and academic attention. One possible approach direction by hill-climbing optimization is an established field, too: see [2][3][4]10,[13][14][15]17,26,29,[36][37][38][39]42] and the tools Breach [13] and S-TaLiRo [4]. We formulate the problem and the methodology, for later use in describing our falsification approach.…”

Section: Hill Climbing-guided Falsificationmentioning

confidence: 99%

“…Since CMA-ES has proved to be the state-of-the-art stochastic algorithm [39], we select CMA-ES as our backend optimizer for the playout phase. 3 We apply the two approaches, ForeSee and Breach, to each benchmark specification reported in Table 1. Since both approaches are based on stochastic optimization, we repeat each experiment for 30 times, as suggested by a guideline for conducting experiments with randomized algorithms [5].…”

Section: Experiments Setupmentioning

confidence: 99%

“…Recently, machine learning techniques have also been applied to falsification to enhance the search ability. For instance, Bayesian optimization [3,11,36] utilizes an acquisition function to balance exploration and exploitation; Reinforcement learning [27,37] naturally emphasizes on exploration.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Effective Hybrid System Falsification Using Monte Carlo Tree Search Guided by QB-Robustness

Zhang

Lyu

Arcaini

et al. 2021

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Hybrid system falsification is an important quality assurance method for cyber-physical systems with the advantage of scalability and feasibility in practice than exhaustive verification. Falsification, given a desired temporal specification, tries to find an input of violation instead of a proof guarantee. The state-of-the-art falsification approaches often employ stochastic hill-climbing optimization that minimizes the degree of satisfaction of the temporal specification, given by its quantitative robust semantics. However, it has been shown that the performance of falsification could be severely affected by the so-called scale problem, related to the different scales of the signals used in the specification (e.g., rpm and speed): in the robustness computation, the contribution of a signal could be masked by another one. In this paper, we propose a novel approach to tackle this problem. We first introduce a new robustness definition, called QB-Robustness, which combines classical Boolean satisfaction and quantitative robustness. We prove that QB-Robustness can be used to judge the satisfaction of the specification and avoid the scale problem in its computation. QB-Robustness is exploited by a falsification approach based on Monte Carlo Tree Search over the structure of the formal specification. First, tree traversal identifies the sub-formulas for which it is needed to compute the quantitative robustness. Then, on the leaves, numerical hill-climbing optimization is performed, aiming to falsify such sub-formulas. Our in-depth evaluation on multiple benchmarks demonstrates that our approach achieves better falsification results than the state-of-the-art falsification approaches guided by the classical quantitative robustness, and it is largely not affected by the scale problem.

show abstract

Section: Hill Climbing-guided Falsificationmentioning

confidence: 99%

Section: Experiments Setupmentioning

confidence: 99%

See 1 more Smart Citation

Effective Hybrid System Falsification Using Monte Carlo Tree Search Guided by QB-Robustness

Zhang

Lyu

Arcaini

et al. 2021

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…For the remainder of this subsection, we will elaborate on the hyper-parametersσ x (line 9 in Algorithm 2 and Equation (8) in Problem 2) andσ θ (line 19 in Algorithm 2 and Equation (10) in Problem 3). Interest in leveraging GPs for the purpose of formal verification, mining, and inference of CPSs has surged as of late [8,13,17,18,28]. Among the current works, many assume that the true hyper-parameters are known a priori and also fixed.…”

Section: Single Satisfactory Valuationmentioning

confidence: 99%

Formal interpretation of cyber-physical system performance with temporal logic

Chen

Sabato

Kong

2018

Cyber-Physical Systems

View full text Add to dashboard Cite

“…We have implemented our MAB-based falsification framework in MATLAB, building on Breach [11]. 3 Our experiments with benchmarks from [7,24,25] demonstrate that our MAB-based approach is a viable one against the scale problem. In particular, our approach is observed to be (almost totally) robust under the change of scaling (i.e.…”

Section: Introductionmentioning

confidence: 99%

Multi-armed Bandits for Boolean Connectives in Hybrid System Falsification

Zhang

Hasuo

Arcaini

2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Hybrid system falsification is an actively studied topic, as a scalable quality assurance methodology for real-world cyber-physical systems. In falsification, one employs stochastic hill-climbing optimization to quickly find a counterexample input to a black-box system model. Quantitative robust semantics is the technical key that enables use of such optimization. In this paper, we tackle the so-called scale problem regarding Boolean connectives that is widely recognized in the community: quantities of different scales (such as speed [km/h] vs. rpm, or worse, rph) can mask each other's contribution to robustness. Our solution consists of integration of the multi-armed bandit algorithms in hill climbing-guided falsification frameworks, with a technical novelty of a new reward notion that we call hill-climbing gain. Our experiments show our approach's robustness under the change of scales, and that it outperforms a state-of-the-art falsification tool.

show abstract

Causality-Aided Falsification

Cited by 8 publications

References 20 publications

Effective Hybrid System Falsification Using Monte Carlo Tree Search Guided by QB-Robustness

Effective Hybrid System Falsification Using Monte Carlo Tree Search Guided by QB-Robustness

Formal interpretation of cyber-physical system performance with temporal logic

Multi-armed Bandits for Boolean Connectives in Hybrid System Falsification

Contact Info

Product

Resources

About