A Survey on Adaptive Random Testing

Huang, Rubing; Sun, Weifeng; Xu, Yinyin; Chen, Haibo; Towey, Dave; Xia, Xin

doi:10.1109/tse.2019.2942921

Cited by 79 publications

(50 citation statements)

References 173 publications

(421 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Without a diverse test suite TS containing a significant proportion of passing and failing test cases, the learning algorithm used in the assumption generation step is not able to accurately infer an assumption. We use search-based testing techniques [12,26,41] for test generation and rely on simulations to run the test cases. Search-based testing allows us to guide the generation of test cases in very large search spaces.…”

Section: Epicurus Overviewmentioning

confidence: 99%

Mining assumptions for software components using machine learning

Gaaloul

Menghi

Nejati

et al. 2020

Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Softw

View full text Add to dashboard Cite

Software verification approaches aim to check a software component under analysis for all possible environments. In reality, however, components are expected to operate within a larger system and are required to satisfy their requirements only when their inputs are constrained by environment assumptions. In this paper, we propose EPIcuRus, an approach to automatically synthesize environment assumptions for a component under analysis (i.e., conditions on the component inputs under which the component is guaranteed to satisfy its requirements). EPIcuRus combines search-based testing, machine learning and model checking. The core of EPIcuRus is a decision tree algorithm that infers environment assumptions from a set of test results including test cases and their verdicts. The test cases are generated using search-based testing, and the assumptions inferred by decision trees are validated through model checking. In order to improve the efficiency and effectiveness of the assumption generation process, we propose a novel test case generation technique, namely Important Features Boundary Test (IFBT), that guides the test generation based on the feedback produced by machine learning. We evaluated EPIcuRus by assessing its effectiveness in computing assumptions on a set of study subjects that include 18 requirements of four industrial models. We show that, for each of the 18 requirements, EPIcuRus was able to compute an assumption to ensure the satisfaction of that requirement, and further, ≈ 78% of these assumptions were computed in one hour.

show abstract

Section: Epicurus Overviewmentioning

confidence: 99%

Mining assumptions for software components using machine learning

Gaaloul

Menghi

Nejati

et al. 2020

Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Softw

View full text Add to dashboard Cite

show abstract

“…As a smarter approach of RT, ART retains the inherent advantages of RT and significantly improves the failure‐detection ability. Owing to the demonstrated effectiveness, it has been deeply studied in the past few years [29, 30 ]. Here, we mainly review some of the studies related to the concerns in this study.…”

Section: Related Workmentioning

confidence: 99%

Adaptive random testing based on flexible partitioning

et al. 2020

Self Cite

View full text Add to dashboard Cite

Adaptive random testing (ART) achieves better failure‐detection effectiveness than random testing due to its even spreading of test cases. ART by random partitioning (RP‐ART) is a lightweight method, but its advantage over random testing is relatively low. Although iterative partition testing (IPT) method has good performance for detecting failures in a block pattern, it loses randomness during the test case generation. To overcome the shortcomings of the above two algorithms, a new algorithm named ART by flexible partitioning (FP‐ART) is proposed. In the FP‐ART, a set of random candidates is used to select an appropriate test case by considering their boundary distance. Accordingly, the corresponding sub‐domain is also partitioned by the new test case. Based on this kind of flexible partitioning, the randomness of test case selection can be guaranteed and the spatial distribution of test cases is even more diverse. According to the results in simulation and empirical experiments, FP‐ART demonstrates better failure‐detection effectiveness than RP‐ART and is more suitable to detect the failures in strip patterns than the IPT method. Meanwhile, its failure‐detection ability is much stronger than that of fixed‐size‐candidate‐set ART in the cases of a relatively high failure rate.

show abstract

“…People have conducted experimental studies on both adaptive random testing and combinatorial testing. In the empirical studies on adaptive random testing, people mainly compare adaptive random testing with pure random testing to verify whether the former testing technique improves fault detection speed [5,6]. In the empirical studies on combinatorial testing, people usually focus on fault detection ability rather than fault detection efficiency [7].…”

Section: Introductionmentioning

confidence: 99%

A Comparison of Fault Detection Efficiency Between Adaptive Random Testing and Greedy Combinatorial Testing for Control Logics in Nuclear Industrial Distributed Control Systems

et al. 2021

View full text Add to dashboard Cite

Due to the complexity of the nuclear industrial distributed control system (DCS), input-domain testing techniques, including random testing and combinatorial testing, are usually utilized to test the control logics in nuclear industrial DCS. To improve the fault detection e of random testing, the adaptive random testing technique selects a test case that significantly differs from all existing test cases. Similarly, to improve the fault detection efficiency of combinatorial testing, the greedy combinatorial testing technique adopts a greedy strategy to generate test cases that cover more uncovered tuple-combinations of parametric values. In this paper, we designed an experiment to compare the fault detection efficiency between adaptive random testing technique and greedy combinatorial testing technique for control logics of nuclear industrial DCS. Through the analysis of the fault detection ratios, the f-measure values, and the values of average percent of faults detected (APFD) on two experimental subjects, including the commonly used benchmarks in the field of Boolean-specification testing as well as a group of Boolean expressions extracted from the control logics in nuclear industrial DCS, the experimental results give us the following conclusions: (1) If the test suites' sizes are relatively small, the fault detection efficiencies of the two techniques are very close though there is a slight advantage in adaptive random testing; (2) With the gradual increase of test suites' sizes, the fault detection efficiency of greedy combinatorial testing is beyond adaptive random testing gradually. Such a result can help us select the appropriate testing techniques in the testing of the control logics in nuclear industry DCS.INDEX TERMS Input-domain testing, fault detection efficiency, adaptive random testing, combinatorial testing, nuclear industrial DCS.

show abstract

A Survey on Adaptive Random Testing

Cited by 79 publications

References 173 publications

Mining assumptions for software components using machine learning

Mining assumptions for software components using machine learning

Adaptive random testing based on flexible partitioning

A Comparison of Fault Detection Efficiency Between Adaptive Random Testing and Greedy Combinatorial Testing for Control Logics in Nuclear Industrial Distributed Control Systems

Contact Info

Product

Resources

About