Estimation of software reliability by stratified sampling

Podgurski, Andy; Masri, Wes; McCleese, Yolanda; Wolff, Francis; Yang, Charles

doi:10.1145/310663.310667

Cited by 60 publications

(35 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Podgurski, et al used cluster analysis of profiles and stratified random sampling to improve the accuracy of software reliability estimates [24]. Leon [5].…”

Section: Related Workmentioning

confidence: 99%

Automated support for classifying software failure reports

Podgurski

León

Francis

et al. 2003

25th International Conference on Software Engineering, 2003. Proceedings.

267

198

View full text Add to dashboard Cite

show abstract

“…Podgurski, et al used cluster analysis of profiles and stratified random sampling to improve the accuracy of software reliability estimates [24]. Leon [5].…”

Section: Related Workmentioning

confidence: 99%

Automated support for classifying software failure reports

Podgurski

León

Francis

et al. 2003

25th International Conference on Software Engineering, 2003. Proceedings.

267

198

View full text Add to dashboard Cite

show abstract

“…Examples of such work include that of ) who demonstrated the advantage of automated clustering of execution profiles over random selection for finding failures by using function caller/callee feature profiles as the basis for cluster formation. This work is in turn based on that of Podgurski et al (1999), who used cluster analysis of profiles and stratified random sampling to calculate estimates of software reliability and found that failures were often isolated in small clusters based on unusual execution profiles. Our work is similar to this and explores the same observed hypothesis about the distribution of failures over clusters, but we investigate the use of test case input/output pairs (and input/output pairs combined with execution profiles) from the system under test instead of execution profiles alone.…”

Section: Test Oracles Based On Anomaly Detection Techniquesmentioning

confidence: 99%

Separating passing and failing test executions by clustering anomalies

Almaghairbe

Roper

2016

Software Qual J

View full text Add to dashboard Cite

Developments in the automation of test data generation have greatly improved efficiency of the software testing process, but the so-called oracle problem (deciding the pass or fail outcome of a test execution) is still primarily an expensive and error-prone manual activity. We present an approach to automatically detect passing and failing executions using cluster-based anomaly detection on dynamic execution data based on firstly, just a system's input/output pairs and secondly, amalgamations of input/output pairs and execution traces. The key hypothesis is that failures will group into small clusters, whereas passing executions will group into larger ones. Evaluation on three systems with a range of faults demonstrates this hypothesis to be valid-in many cases small clusters were composed of at least 60 % failures (and often more). Concentrating the failures in these small clusters substantially reduces the numbers of outputs that a developer would need to manually examine following a test run and illustrates that the approach has the potential to improve the effectiveness and efficiency of the testing process.

show abstract

“…These clusterings can be sampled in different ways to find faults, analyse a fault in detail to come up with comprehensive fixes and so on. Earlier works have applied clustering techniques to improve several testing aspects including observation-based testing [2]- [4], [16], regression test selection [6] and test suite minimization and prioritization [17], [7].…”

Section: Related Workmentioning

confidence: 99%

Test Cluster Selection Using Cover Coefficients

Subramaniam¹,

Chundi²

2015

Software Engineering

View full text Add to dashboard Cite

Clustering test profiles to retrieve relevant tests is a recurring theme in software validation. A novel clustering approach using a probabilistic notion of coverage among line-based test profiles is described. The approach automatically determines the number of clusters to generate a clustering and can potentially group together tests to execute a few distinct lines of code. A simple method to identify tests affected by program changes is developed and used to determine the retrieval effort needed for cluster-based retrieval of affected tests. The approach is applied to four unix utility programs from a popular testing benchmark. Our results show that comparing original and new clusterings with respect to test cases is a promising criterion for deciding the potential re-clustering points in software evolution.

show abstract

Estimation of software reliability by stratified sampling

Cited by 60 publications

References 25 publications

Automated support for classifying software failure reports

Automated support for classifying software failure reports

Separating passing and failing test executions by clustering anomalies

Test Cluster Selection Using Cover Coefficients

Contact Info

Product

Resources

About