NOTICE: A Framework for Non-Functional Testing of Compilers

Boussaa, Mohamed; Barais, Olivier; Baudry, Benoît; Sunyé, Gerson

doi:10.1109/qrs.2016.45

Cited by 5 publications

(5 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Compiler testing is a difficult and essential task due to the complexity and importance of compilers, and thus it has attracted extensive attentions from researchers [37], [38], [39], [40], [41]. Research on compiler testing can be divided into three main aspects, i.e., test-program generation, testoracle construction, and test-execution optimization.…”

Section: Related Work 81 Compiler Testingmentioning

confidence: 99%

Coverage Prediction for Accelerating Compiler Testing

Chen

Wang

Hao

et al. 2021

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

Compilers are one of the most fundamental software systems. Compiler testing is important for assuring the quality of compilers. Due to the crucial role of compilers, they have to be well tested. Therefore, automated compiler testing techniques (those based on randomly generated programs) tend to run a large number of test programs (which are test inputs of compilers). The cost for compilation and execution for these test programs is significant. These techniques can take a long period of testing time to detect a relatively small number of compiler bugs. That may cause many practical problems, e.g., bringing a lot of costs including time costs and financial costs, and delaying the development/release cycle. Recently, some approaches have been proposed to accelerate compiler testing by executing test programs that are more likely to trigger compiler bugs earlier according to some criteria. However, these approaches ignore an important aspect in compiler testing: different test programs may have similar test capabilities (i.e., testing similar functionalities of a compiler, even detecting the same compiler bug), which may largely discount their acceleration effectiveness if the test programs with similar test capabilities are executed all the time. Test coverage is a proper approximation to help distinguish them, but collecting coverage dynamically is infeasible in compiler testing since most test programs are generated on the fly by automatic test-generation tools like Csmith. In this paper, we propose the first method to predict test coverage statically for compilers, and then propose to prioritize test programs by clustering them according to the predicted coverage information. The novel approach to accelerating compiler testing through coverage prediction is called COP (short for COverage Prediction). Our evaluation on GCC and LLVM demonstrates that COP significantly accelerates compiler testing, achieving an average of 51.01% speedup in test execution time on an existing dataset including three old release versions of the compilers and achieving an average of 68.74% speedup on a new dataset including 12 latest release versions. Moreover, COP outperforms the state-of-the-art acceleration approach significantly by improving 17.16%∼82.51% speedups in different settings on average.

show abstract

Section: Related Work 81 Compiler Testingmentioning

confidence: 99%

Coverage Prediction for Accelerating Compiler Testing

Chen

Wang

Hao

et al. 2021

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

show abstract

“…At present, and as discussed above, the most widely used compilation optimization goal is the acceleration ratio of the target code [19] [20] [21], with code size [22] [23] also being a relatively common optimization goal. Other optimization goals include CPU consumption [24], power efficiency [25], and resilience [26]. However, as mentioned in the introduction, there is still limited research on program reliability as an optimization goal.…”

Section: Related Workmentioning

confidence: 99%

Compilation Optimization Pass Selection Using Gate Graph Attention Neural Network for Reliability Improvement

Meng

et al. 2020

IEEE Access

View full text Add to dashboard Cite

When dealing with different programs or applications, it is necessary to select the appropriate compilation optimization pass or subsequence for the program. Machine learning is widely used as an efficient technological means of solving this problem. However, the most important problem when using machine learning is the extraction of program features. Obtaining more semantic and syntax information and complex transitions among code segments from the source code are obviously necessary in this context, and is also an area that may have been neglected by previous work. Ensuring the integrity and effectiveness of program information is key to this problem. Moreover, when performing and improving the selection, the measurement indicators are often program performance, code size, etc.; there is limited research on program reliability in this context, which requires both the longest measurement time and the most complicated measurement methods. Accordingly, this paper establishes a combined program feature extraction model and proposes a graph-based compilation optimization pass selection model that learns heuristics for program reliability. This experiment was performed using the clang compilation framework. The alternative compilation optimization pass adopts the C language standard compilation optimization passes. Compared with traditional machine learning methods, our model improves the average accuracy by between 5% and 11% in the optimization pass selection for program reliability. Our experiments also demonstrate the strong scalability of our proposed model. INDEX TERMS Compilation optimization, pass selection, extraction of program features, GGNN-based model, program reliability.

show abstract

“…Ansel 等 [13] 提出了一个为程序满足多个目标而选择编译优化序列的开源框架 "OpenTuner", 该框架使用了包括遗传算法在内的多种进化算法, 其在 6 个项目和 16 个基准测试集上的实验表明该框架效果良好, 能够获得最高 2.8 倍的加速比. Boussaa 等 [14] 在使用遗传算法的基础上, 利用一种新的指标来表示种群中个体的稀疏性, 在每次迭代选择时尽量确保种群中个体的多样性, 进而为待编译程序选择合适的编译优化序列.…”

Section: 启发式搜索算法使用启发式的方法在编译优化选项组合空间中搜索最优的编译优化序列比如unclassified

“…另一个受到研究人员关注的优化目标是目标代码的规模 [27,28] . 目标代码规模对于程序编译是一个非常重要的优化目标, 尤除了将目标代码的执行速度和代码规模作为优化目标之外, 研究者还分析了 CPU 使用 (CPU consumption) [14] 、功耗 (power efficiency) [29] 以及差错恢复能力 (resilience) [30] 等目标, 以满足特定情况下某些程序的需求. 比如, Sangchoolie 等 [31] 分析了在使用编译器 GCC 提供的各个标准编译优化序列 (-O0, -O1, -O2, -O3 等) 编译待编译程序的差错恢复能力.…”

Section: 编译优化目标unclassified

“…经验研究是对某些问题的分析, 例如文献 [32] 分析了静态特征、动态特征等对于机器学习算法在应用于编译优化序列选择时的影响. 评价研究是对已实现的方法、框架评价总结, 文献 [14,19] 分别总结分析了两个编译优化序列选择框架 NOTICE [19] 和 Milepost GCC [14] . 个人观点是指作者对于某些问题的个人想法, 如 Nazarian 等 [33] 通过总结分析优化能量消耗和优化程序可靠性两类研究工作中的各类方法, 针对编译优化能量消耗和程序可靠性的兼容性提出了自己的观点.…”

Section: 编译优化目标unclassified

See 1 more Smart Citation

Selection of compiler-optimization sequences

Gao

Zhang

et al. 2019

Sci. Sin.-Inf.

View full text Add to dashboard Cite

Multi-objective optimization of feature selection using hybrid cat swarm optimization SCIENCE CHINA Technological Sciences Multiobjective optimization using an immunodominance and clonal selection inspired algorithm Science in China Series F-Information Sciences 51, 1064 (2008); OpenMP compiler for distributed memory architectures SCIENCE CHINA Information Sciences 53, 932 (2010); MPtostream: an OpenMP compiler for CPU-GPU heterogeneous parallel systems SCIENCE CHINA Information Sciences 55, 1961 (2012); A bi-objective optimization approach for selection of passive energy alternatives in retrofit projects under cost uncertainty Energy and Built Environment 1, 77 (2020);

show abstract

NOTICE: A Framework for Non-Functional Testing of Compilers

Cited by 5 publications

References 28 publications

Coverage Prediction for Accelerating Compiler Testing

Coverage Prediction for Accelerating Compiler Testing

Compilation Optimization Pass Selection Using Gate Graph Attention Neural Network for Reliability Improvement

Selection of compiler-optimization sequences

Contact Info

Product

Resources

About