Automatic Unit Test Generation for Machine Learning Libraries: How Far Are We?

Wang, Song; Shrestha, Nishtha; Subburaman, Abarna Kucheri; Wang, Junjie; Wei, Moshi; Nagappan, Nachiappan

doi:10.1109/icse43902.2021.00138

Cited by 29 publications

(11 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[134], [46], [48]-[51] 2 Clustering [10], [12], [42], [43], [53], [57], [60], [61], [69], [71], [79], [85], [21], [89], [90], [93], [101]- [103], [111], [114], [117], [118], [25], [126], [129], [130], [132], [133], [27], [31]- [35] 35…”

Section: Discussionmentioning

confidence: 99%

Trend Application of Machine Learning in Test Case Prioritization: A Review on Techniques

et al. 2021

View full text Add to dashboard Cite

Software quality can be assured by passing the process of software testing. However, software testing process involve many phases which lead to more resources and time consumption. To reduce these downsides, one of the approaches is to adopt test case prioritization (TCP) where numerous works has indicated that TCP do improve the overall software testing performance. TCP does have several kinds of techniques which have their own strengths and weaknesses. As for this review paper, the main objective of this paper is to examine deeper on machine learning (ML) techniques based on research questions created. The research method for this paper was designed in parallel with the research questions. Consequently, 110 primary studies were selected where, 58 were journal articles, 50 were conference papers and 2 considered as others articles. For overall result, it can be said that ML techniques in TCP has trending in recent years yet some improvements are certainly welcomed. There are multiple ML techniques available, in which each technique has specified potential values, advantages, and limitation. It is notable that ML techniques has been considerably discussed in TCP approach for software testing.

show abstract

Section: Discussionmentioning

confidence: 99%

Trend Application of Machine Learning in Test Case Prioritization: A Review on Techniques

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Miller et al [34] proposed the first SBST technique to generate test data for functions with inputs of float type. SBST techniques have been widely used in various objects under test [3,10,13,14,17,23,32,47,52], and types of software testing [29,45,51]. Most researchers focus on (1) search algorithms: Tonella [48] proposed to iterate to generate one test case for each branch.…”

Section: Related Workmentioning

confidence: 99%

Selectively Combining Multiple Coverage Goals in Search-Based Unit Test Generation

Zhou¹,

Zhou²,

Fang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Unit testing is a critical part of software development process, ensuring the correctness of basic programming units in a program (e.g., a method). Search-based software testing (SBST) is an automated approach to generating test cases. SBST generates test cases with genetic algorithms by specifying the coverage criterion (e.g., branch coverage). However, a good test suite must have different properties, which cannot be captured by using an individual coverage criterion. Therefore, the state-of-the-art approach combines multiple criteria to generate test cases. As combining multiple coverage criteria brings multiple objectives for optimization, it hurts the test suites' coverage for certain criteria compared with using the single criterion. To cope with this problem, we propose a novel approach named smart selection. Based on the coverage correlations among criteria and the coverage goals' subsumption relationships, smart selection selects a subset of coverage goals to reduce the number of optimization objectives and avoid missing any properties of all criteria. We conduct experiments to evaluate smart selection on 400 Java classes with three state-of-the-art genetic algorithms. On average, smart selection outperforms combining all goals on 65.1% of the classes having significant differences between the two approaches. CCS CONCEPTS• Software and its engineering → Search-based software engineering; Software testing and debugging.

show abstract

“…Beyond quantum computing platforms, other widely used platforms are subject to testing and verification approaches, e.g., in the form of compiler testing [Chen et al 2020;Le et al 2014Le et al , 2015Yang et al 2011;Zhang et al 2017], compiler verification [Leroy 2009], or automated testing of deep learning libraries [Pham et al 2019;Wang et al 2021a].…”

Section: Correctness Of Quantum Computing and Other Platformsmentioning

confidence: 99%

Bugs in Quantum Computing Platforms: An Empirical Study

Paltenghi¹,

Pradel²

2021

Preprint

View full text Add to dashboard Cite

The interest in quantum computing is growing, and with it, the importance of software platforms to develop quantum programs. Ensuring the correctness of such platforms is important, and it requires a thorough understanding of the bugs they typically suffer from. To address this need, this paper presents the first in-depth study of bugs in quantum computing platforms. We gather and inspect a set of 223 real-world bugs from 18 open-source quantum computing platforms. Our study shows that a significant fraction of these bugs (39.9%) are quantum-specific, calling for dedicated approaches to prevent and find them. The bugs are spread across various components, but quantum-specific bugs occur particularly often in components that represent, compile, and optimize quantum programming abstractions. Many quantum-specific bugs manifest through unexpected outputs, rather than more obvious signs of misbehavior, such as crashes. Finally, we present a hierarchy of recurrent bug patterns, including ten novel, quantum-specific patterns. Our findings not only show the importance and prevalence bugs in quantum computing platforms, but they help developers to avoid common mistakes and tool builders to tackle the challenge of preventing, finding, and fixing these bugs.

show abstract

Automatic Unit Test Generation for Machine Learning Libraries: How Far Are We?

Cited by 29 publications

References 56 publications

Trend Application of Machine Learning in Test Case Prioritization: A Review on Techniques

Trend Application of Machine Learning in Test Case Prioritization: A Review on Techniques

Selectively Combining Multiple Coverage Goals in Search-Based Unit Test Generation

Bugs in Quantum Computing Platforms: An Empirical Study

Contact Info

Product

Resources

About