Learning-based testing for autonomous systems using spatial and temporal requirements

Khosrowjerdi, Hojat; Michaëlsson, Karl

doi:10.1145/3243127.3243129

Cited by 18 publications

(8 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The important thing is that the found studies are a good representation of the population, which we ensured in this study by adopting a rigorous paper selection process. All Stages Approach/Technique/Method RNN [63] All Stages Other Other [64] All Stages Other Other [56] All Stages Comparative Analysis Other [57] All Stages Other Other [69] All Stages Other RNN, RBM [14] All Stages Model/Framework Other [67] All Stages Tool RF [49] All Stages Other Other [62] All Stages Tool NLP [61] All Stages Other DT [48] All Stages Other Other [65] All Stages Other Other [1] All Stages Other Other [60] All Stages Other Other [68] All Stages Comparative Analysis LR, SVM, NB [66] All Stages Approach/Technique/Method LSTM [11] All Stages Other Other [59] All Stages Other Other [21] Requirements Approach/Technique/Method NB, KNN, RF [70] Requirements Approach/Technique/Method SVM, SMO, NB [82] Requirements Approach/Technique/Method PN [75] Requirements Model/Framework ProbPoly [71] Requirements Approach/Technique/Method Text2Model [84] Requirements Approach/Technique/Method RF [72] Requirements Approach/Technique/Method Other [22] Requirements Approach/Technique/Method NB, RF, LR, SGD, DT [23] Requirements Approach/Technique/Method Boosting [81] Requirements Approach/Technique/Method NSGA-II algorithm [24] Requirements Model/Framework CNN [73] Requirements Approach/Technique/Method FL [76] Requirements Approach/Technique/Method LP, SMO, NB, KNN [86] Requirements Approach/Technique/Method J48, FSS, CFS [25] Requirements Model/Framework RNN [85] Requirements Model/Framework KNN [...…”

Section: A External Validitymentioning

confidence: 99%

A Literature Review of Using Machine Learning in Software Development Life Cycle Stages

Shafiq¹,

Mashkoor

Mayr-Dorn

et al. 2021

IEEE Access

View full text Add to dashboard Cite

The software engineering community is rapidly adopting machine learning for transitioning modern-day software towards highly intelligent and self-learning systems. However, the software engineering community is still discovering new ways how machine learning can offer help for various software development life cycle stages. In this article, we present a study on the use of machine learning across various software development life cycle stages. The overall aim of this article is to investigate the relationship between software development life cycle stages, and machine learning tools, techniques, and types. We attempt a holistic investigation in part to answer the question of whether machine learning favors certain stages and/or certain techniques.

show abstract

Section: A External Validitymentioning

confidence: 99%

A Literature Review of Using Machine Learning in Software Development Life Cycle Stages

Shafiq¹,

Mashkoor

Mayr-Dorn

et al. 2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Black-box checking (BBC) [32], which is also known as learning-based testing (LBT), is an application of active automata learning for system testing. BBC has been used for testing numerical software [28], distributed systems [29], and autonomous systems [23]. BBC is implemented in LBTest [30] and LearnLib [22,27].…”

Section: Related Workmentioning

confidence: 99%

Efficient Black-Box Checking via Model Checking with Strengthened Specifications

Shijubo¹,

Waga²,

Suenaga³

2021

Preprint

View full text Add to dashboard Cite

Black-box checking (BBC) is a testing method for cyberphysical systems (CPSs) as well as software systems. BBC consists of active automata learning and model checking; a Mealy machine is learned from the system under test (SUT), and the learned Mealy machine is verified against a specification using model checking. When the Mealy machine violates the specification, the model checker returns an input witnessing the specification violation of the Mealy machine. We use it to refine the Mealy machine or conclude that the SUT violates the specification. Otherwise, we conduct equivalence testing to find an input witnessing the difference between the Mealy machine and the SUT. In the BBC for CPSs, equivalence testing tends to be time-consuming due to the time for the system execution. In this paper, we enhance the BBC utilizing model checking with strengthened specifications. By model checking with a strengthened specification, we have more chance to obtain an input witnessing the specification violation than model checking with the original specification. The refinement of the Mealy machine with such an input tends to reduce the number of equivalence testing, which improves the efficiency. We conducted experiments with an automotive benchmark. Our experiment results demonstrate the merit of our method.

show abstract

“…A great effort has been devoted to a more practical direction of BBC, including the testing of automotive systems. For example, case studies on testing of automotive software systems are shown in [32] and an application to the CPSs with continuous dynamics is presented in [31,39]. However, up to our knowledge, there is no work exploiting the quantitative satisfaction degree of the requirements in addition to Boolean satisfaction.…”

Section: Related Workmentioning

confidence: 99%

Falsification of cyber-physical systems with robustness-guided black-box checking

Waga

2020

Proceedings of the 23rd International Conference on Hybrid Systems: Computation and Control

View full text Add to dashboard Cite

For exhaustive formal verification, industrial-scale cyberphysical systems (CPSs) are often too large and complex, and lightweight alternatives (e.g., monitoring and testing) have attracted the attention of both industrial practitioners and academic researchers. Falsification is one popular testing method of CPSs utilizing stochastic optimization. In stateof-the-art falsification methods, the result of the previous falsification trials is discarded, and we always try to falsify without any prior knowledge. To concisely memorize such prior information on the CPS model and exploit it, we employ Black-box checking (BBC), which is a combination of automata learning and model checking. Moreover, we enhance BBC using the robust semantics of STL formulas, which is the essential gadget in falsification. Our experiment results suggest that our robustness-guided BBC outperforms a state-of-the-art falsification tool. CCS CONCEPTS• Computer systems organization → Embedded and cyber-physical systems; Real-time system specification; • Software and its engineering → Formal software verification; Search-based software engineering.

show abstract

Learning-based testing for autonomous systems using spatial and temporal requirements

Cited by 18 publications

References 26 publications

A Literature Review of Using Machine Learning in Software Development Life Cycle Stages

A Literature Review of Using Machine Learning in Software Development Life Cycle Stages

Efficient Black-Box Checking via Model Checking with Strengthened Specifications

Falsification of cyber-physical systems with robustness-guided black-box checking

Contact Info

Product

Resources

About