Assessing test artifact quality—A tertiary study

Tran, Huynh Khanh Vi; Unterkalmsteiner, Michael; Börstler, Jürgen; Ali, Nauman bin

doi:10.1016/j.infsof.2021.106620

Cited by 14 publications

(33 citation statements)

References 100 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…1 Table A. 5 shows the secondary and primary studies that were the subject of the discussion of overlaps in Section 3. The paper IDs for the secondary studies map those in Table A.5.…”

Section: Data Availabilitymentioning

confidence: 99%

Double-counting in software engineering tertiary studies — An overlooked threat to validity

Börstler

Ali

Petersen

2023

Information and Software Technology

Self Cite

View full text Add to dashboard Cite

“…1 Table A. 5 shows the secondary and primary studies that were the subject of the discussion of overlaps in Section 3. The paper IDs for the secondary studies map those in Table A.5.…”

Section: Data Availabilitymentioning

confidence: 99%

Double-counting in software engineering tertiary studies — An overlooked threat to validity

Börstler

Ali

Petersen

2023

Information and Software Technology

Self Cite

View full text Add to dashboard Cite

“…Our case study focuses on a quantitative comparison between manually and automatically created test cases. However, several other metrics are available to benchmark test cases [45]. For example, structural criteria like test understandability investigate whether a test is easy to understand in terms of its internal and external descriptions.…”

Section: Key Take-awaymentioning

confidence: 99%

Automatic Creation of Acceptance Tests by Extracting Conditionals from Requirements: NLP Approach and Case Study

Fischbach¹,

Frattini²,

Vogelsang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Acceptance testing is crucial to determine whether a system fulfills end-user requirements. However, the creation of acceptance tests is a laborious task entailing two major challenges: (1) practitioners need to determine the right set of test cases that fully covers a requirement, and (2) they need to create test cases manually due to insufficient tool support. Existing approaches for automatically deriving test cases require semi-formal or even formal notations of requirements, though unrestricted natural language is prevalent in practice. In this paper, we present our tool-supported approach CiRA (Conditionals in Requirements Artifacts) capable of creating the minimal set of required test cases from conditional statements in informal requirements. We demonstrate the feasibility of CiRA in a case study with three industry partners. In our study, out of 578 manually created test cases, 71.8 % can be generated automatically. Additionally, CiRA discovered 80 relevant test cases that were missed in manual test case design. CiRA is publicly available at www.cira.bth.se/demo/.

show abstract

“…In a recent tertiary study [25] on test artifact quality, as suggested by Kitchenham et al [18], we constructed a QGS by collecting relevant papers from an earlier tertiary study with a related broader topic [13] (software testing). Our assumption was that a tertiary review of software testing research, in general, would also cover secondary studies on the relatively narrower topic of test artifact quality.…”

Section: Introductionmentioning

confidence: 99%

“…Section 3 presents the related work and our contribution. Section 4 summarizes the search process and search validation in our tertiary study [25]. Section 5 presents our findings when comparing the search results between the two tertiary studies [13,25].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

How good are my search strings? Reflections on using an existing review as a quasi-gold standard

Tran¹,

Börstler²,

Ali³

et al. 2022

EISEJ

View full text Add to dashboard Cite

Background: Systematic literature studies (SLS) have become a core research methodology in Evidence-based Software Engineering (EBSE). Search completeness, i.e., finding all relevant papers on the topic of interest, has been recognized as one of the most commonly discussed validity issues of SLSs. Aim: This study aims at raising awareness on the issues related to search string construction and on search validation using a quasi-gold standard (QGS). Furthermore, we aim at providing guidelines for search string validation. Method: We use a recently completed tertiary study as a case and complement our findings with the observations from other researchers studying and advancing EBSE. Results: We found that the issue of assessing QGS quality has not seen much attention in the literature, and the validation of automated searches in SLSs could be improved. Hence, we propose to extend the current search validation approach by the additional analysis step of the automated search validation results and provide recommendations for the QGS construction. Conclusion: In this paper, we report on new issues which could affect search completeness in SLSs. Furthermore, the proposed guideline and recommendations could help researchers implement a more reliable search strategy in their SLSs.

show abstract

Assessing test artifact quality—A tertiary study

Cited by 14 publications

References 100 publications

Double-counting in software engineering tertiary studies — An overlooked threat to validity

Double-counting in software engineering tertiary studies — An overlooked threat to validity

Automatic Creation of Acceptance Tests by Extracting Conditionals from Requirements: NLP Approach and Case Study

How good are my search strings? Reflections on using an existing review as a quasi-gold standard

Contact Info

Product

Resources

About