Assessing the Test Suite of a Large System Based on Code Coverage, Efficiency and Uniqueness

Vidács, László; Horváth, Ferenc; Tengeri, Dávid; Beszedes, Arpad

doi:10.1109/saner.2016.69

Cited by 5 publications

(7 citation statements)

References 11 publications

(15 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Vidacs et al [26] uses the code coverage, efficiency & uniqueness for Assessing the Test suites of large system. The authors argue that better coverage or partitioning can be achieved using more test cases, provided test cases are different.…”

Section: ) Related Measures Presented In the Literaturementioning

confidence: 99%

See 1 more Smart Citation

Regression Testing Goals - View of Practitioners and Researchers

Minhas

Petersen

Ali

et al. 2017

2017 24th Asia-Pacific Software Engineering Conference Workshops (APSECW)

View full text Add to dashboard Cite

Context: Regression testing is a well-researched area. However, the majority regression testing techniques proposed by the researchers are not getting the attention of the practitioners. Communication gaps between industry and academia, and disparity in the regression testing goals are the main reasons. Close collaboration can help in bridging the communication gaps and resolving the disparities. Objective: The study aims at exploring the views of academics and practitioners about the goals of regression testing. The purpose is to investigate the commonalities and differences in their viewpoints and defining some common goals for the success of regression testing. Method: We conducted a focus group study, with 7 testing experts from industry and academia. 4 testing practitioners from 2 companies and 3 researchers from 2 universities participated in the study. We followed GQM approach, to elicit the regression testing goals, information needs, and measures. Results: 43 regression testing goals were identified by the participants, which were reduced to 10 on the basis of similarity among the identified goals. Later during the priority assignment process, 5 goals were discarded, because the priority assigned to these goals was very low. Participants identified 47 information needs/questions required to evaluate the success of regression testing with reference to goal G5 (confidence). Which were then reduced to 10 on the basis of similarity. Finally, we identified measures to gauge those information needs/questions, which were corresponding to the goal (G5). Conclusions: We observed that participation level of practitioners and researchers during the elicitation of goals and questions was same. We found a certain level of agreement between the participants regarding the regression testing definitions and goals. But there was some level of disagreement regarding the priorities of the goals. We also identified the need to implement a regression testing evaluation framework in the participating companies.

show abstract

Section: ) Related Measures Presented In the Literaturementioning

confidence: 99%

“…The key constraint of regression testing is the maintenance of the regression test suite (adding new test cases or updating or deleting obsolete test cases) [20], [26]. Test suite maintenance is not an easy task and if not done in a correct manner, utility of the test suite will be decreased and associated risks will be amplified [12].…”

Section: Introductionmentioning

confidence: 99%

Regression Testing Goals - View of Practitioners and Researchers

Minhas

Petersen

Ali

et al. 2017

2017 24th Asia-Pacific Software Engineering Conference Workshops (APSECW)

View full text Add to dashboard Cite

show abstract

“…Tengeri et al (2015) proposed an approach to group related test and code elements together, but this was based on manual classification. In the method, various metrics are computed and used as general indicators of test suite quality, and later it has been applied in a deep analysis of the WebKit system (Vidács et al 2016).…”

Section: Clustering In Softwarementioning

confidence: 99%

Differences between a static and a dynamic test-to-code traceability recovery method

et al. 2018

View full text Add to dashboard Cite

Recovering test-to-code traceability links may be required in virtually every phase of development. This task might seem simple for unit tests thanks to two fundamental unit testing guidelines: isolation (unit tests should exercise only a single unit) and separation (they should be placed next to this unit). However, practice shows that recovery may be challenging because the guidelines typically cannot be fully followed. Furthermore, previous works have already demonstrated that fully automatic test-to-code traceability recovery for unit tests is virtually impossible in a general case. In this work, we propose a semi-automatic method for this task, which is based on computing traceability links using static and dynamic approaches, comparing their results and presenting the discrepancies to the user, who will determine the final traceability links based on the differences and contextual information. We define a set of discrepancy patterns, which can help the user in this task. Additional outcomes of analyzing the discrepancies are structural unit testing issues and related refactoring suggestions. For the static test-to-code traceability, we rely on the physical code structure, while for the dynamic, we use code coverage information. In both cases, we compute combined test and code clusters which represent sets of mutually traceable elements. We also present an empirical study of the method involving 8 non-trivial open source Java systems.

show abstract

“…We proposed a related approach to group related test and code elements together, but this was based on manual classification done by the testers and developers [20]. In the method, various metrics are computed and used as general indicators of test suite quality, and later it has been applied in a deep analysis of the WebKit system [21].…”

Section: Related Workmentioning

confidence: 99%

Are My Unit Tests in the Right Package?

Balogh

Gergely

Beszedes

et al. 2016

2016 IEEE 16th International Working Conference on Source Code Analysis and Manipulation (SCAM)

Self Cite

View full text Add to dashboard Cite

The software development industry has adopted written and de facto standards for creating effective and maintainable unit tests. Unfortunately, like any other source code artifact, they are often written without conforming to these guidelines, or they may evolve into such a state. In this work, we address a specific type of issues related to unit tests. We seek to automatically uncover violations of two fundamental rules: 1) unit tests should exercise only the unit they were designed for, and 2) they should follow a clear packaging convention. Our approach is to use code coverage to investigate the dynamic behaviour of the tests with respect to the code elements of the program, and use this information to identify highly correlated groups of tests and code elements (using community detection algorithm). This grouping is then compared to the trivial grouping determined by package structure, and any discrepancies found are treated as "bad smells." We report on our related measurements on a set of large open source systems with notable unit test suites, and provide guidelines through examples for refactoring the problematic tests.

show abstract

Assessing the Test Suite of a Large System Based on Code Coverage, Efficiency and Uniqueness

Cited by 5 publications

References 11 publications

Regression Testing Goals - View of Practitioners and Researchers

Regression Testing Goals - View of Practitioners and Researchers

Differences between a static and a dynamic test-to-code traceability recovery method

Are My Unit Tests in the Right Package?

Contact Info

Product

Resources

About