Enhanced compiler bug isolation via memoized search

Chen, Junjie; Ma, Haoyang; Zhang, Lingming

doi:10.1145/3324884.3416570

Cited by 26 publications

(3 citation statements)

References 71 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Test program generation. Existing test program generation approaches can be divided into two categories: test-program generation from scratch [8,23,36,42,62] and test program generation by mutating existing test programs [18,21,40,41,55,67,68]. Regarding the former, one of the most widely-used C test-program generation tools is Csmith, which was proposed and developed by Yang et al [62] and has been introduced in Section 2.1.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Boosting Compiler Testing via Compiler Optimization Exploration

Chen

Suo

2022

ACM Trans. Softw. Eng. Methodol.

Self Cite

View full text Add to dashboard Cite

Compilers are a kind of important software, and similar to the quality assurance of other software, compiler testing is one of the most widely-used ways of guaranteeing their quality. Compiler bugs tend to occur in compiler optimizations. Detecting optimization bugs needs to consider two main factors: 1) the optimization flags controlling the accessability of the compiler buggy code should be turned on; and 2) the test program should be able to trigger the buggy code. However, existing compiler testing approaches only consider the latter to generate effective test programs, but just run them under several pre-defined optimization levels (e.g., -O0 , -O1 , -O2 , -O3 , -Os in GCC). To better understand the influence of compiler optimizations on compiler testing, we conduct the first empirical study, and find that 1) all the bugs detected under the widely-used optimization levels are also detected under the explored optimization settings (we call a combination of optimization flags turned on for compilation an optimization setting ), while 83.54% of bugs are only detected under the latter; 2) there exist both inhibition effect and promotion effect among optimization flags for compiler testing, indicating the necessity and challenges of considering the factor of compiler optimizations in compiler testing. We then propose the first approach, called COTest , by considering both factors to test compilers. Specifically, COTest first adopts machine learning (the XGBoost algorithm) to model the relationship between test programs and optimization settings, to predict the bug-triggering probability of a test program under an optimization setting. Then, it designs a diversity augmentation strategy to select a set of diverse candidate optimization settings for prediction for a test program. Finally, Top-K optimization settings are selected for compiler testing according to the predicted bug-triggering probabilities. The experiments on GCC and LLVM demonstrate its effectiveness, especially COTest detects 17 previously unknown bugs, 11 of which have been fixed or confirmed by developers.

show abstract

Section: Related Workmentioning

confidence: 99%

“…In particular, EMI has three instantiations, including Orion [40], Athena [41], and Hermes [55]. Also, Chen et al [18,21] proposed to generate passing test programs by mutating a given bug-triggering test program for compiler bug isolation.…”

Section: Related Workmentioning

confidence: 99%

Boosting Compiler Testing via Compiler Optimization Exploration

Chen

Suo

2022

ACM Trans. Softw. Eng. Methodol.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Deep Learning (DL) has been playing a significant role in various application domains, including image classification [37,58], natural language processing [31,33], game playing [57], and software engineering [22,42,70]. Through such applications, DL has substantially improved our daily life [19,34,56,60,67].…”

Section: Introductionmentioning

confidence: 99%

Free Lunch for Testing: Fuzzing Deep-Learning Libraries from Open Source

Wei¹,

Yinlin²,

Yang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Deep learning (DL) systems can make our life much easier, and thus is gaining more and more attention from both academia and industry. Meanwhile, bugs in DL systems can be disastrous, and can even threaten human lives in safety-critical applications. To date, a huge body of research efforts have been dedicated to testing DL models. However, interestingly, there is still limited work for testing the underlying DL libraries, which are the foundation for building, optimizing, and running the DL models. One potential reason is that test generation for the underlying DL libraries can be rather challenging since their public APIs are mainly exposed in Python, making it even hard to automatically determine the API input parameter types due to dynamic typing. In this paper, we propose FreeFuzz, the first approach to fuzzing DL libraries via mining from open source. More specifically, FreeFuzz obtains code/models from three different sources: 1) code snippets from the library documentation, 2) library developer tests, and 3) DL models in the wild. Then, FreeFuzz automatically runs all the collected code/models with instrumentation to trace the dynamic information for each covered API, including the types and values of each parameter during invocation, and shapes of input/output tensors. Lastly, FreeFuzz will leverage the traced dynamic information to perform fuzz testing for each covered API. The extensive study of FreeFuzz on PyTorch and TensorFlow, two of the most popular DL libraries, shows that FreeFuzz is able to automatically trace valid dynamic information for fuzzing 1158 popular APIs, 9X more than state-of-the-art LEMON with 3.5X lower overhead than LEMON. Furthermore, FreeFuzz is able to detect 35 bugs for PyTorch and TensorFlow (with 31 confirmed by developers and 30 previously unknown).

show abstract