What Do We Know about Alignment of Requirements Engineering and Software Testing?

Abstract

Context:The alignment of different software engineering activities for coordinated functioning and optimized product development is of great importance, particularly in industrialscale development. The link between intermediate activities has been researched extensively, but the link between requirements engineering (RE) and software testing (ST) is a relatively less explored area. Objective: The objective of this study is to aggregate, structure, and classify all existing research regarding alignment of RE and ST published by the end of 2015. Method: We conducted a systematic mapping study (SMS) and aggregated all studies relevant to our scope. The primary studies are analyzed in terms of publication trend, focus area, i.e., how alignment is supported, the application domain and benefits and challenges, methodological data, and scientific rigor and industrial relevance. Results: There is a growing interest towards the topic. Several different techniques have been identified to improve RE and ST alignment. Test generation from requirements specification has received most attention. Alignment of RE and ST is particularly important for large safety-critical domains. While many challenges have been reported, the supporting evidence for benefits is scarce. Frameworks/methods/techniques is the most frequent contribution type. Solution proposal and evaluation research were the most frequently applied research type. Case study research was the most frequently applied research method, however, almost half of the studies did not clearly report any research method. Conclusion: Despite the numerous approaches that are proposed, it is not clear what approach is suitable in what context and why. To support industry in RE and ST alignment, guidelines and tool support are needed. The supporting evidence for claimed benefits is very limited. Overall, the research area is in its early stages and an increase in both the number and rigor of empirical studies are required.

Strategies to manage quality requirements in agile software development: a multiple case study

“…In cases of large systems with complex dependencies in which some level of hierarchy is need for decision making, there is an increased general need to pay careful attention to aspects such as documentation, communication, and traceability, as well as to generally and carefully align RE with testing through people, processes, and artifacts. This alignment has been the interest of many large software development organizations in the automotive and avionic domains (Karhapää et al 2017), and QR management could benefit from it.…”

Summary of the 4th International Workshop on Requirements Engineering and Testing (RET 2017)

“…The second session started with an invited talk on a systematic mapping study that reviewed the scientific literature addressing requirements engineering and software testing alignment. Karhapää presented his work, previously published at the 21st Conference on Evaluation and Assessment in Software Engineering (EASE 2017) [4], in which the authors identified 7 focus areas in the 80 reviewed publications. Prior to the workshop, we used their structure to create a visual map of research on RET alignment, including their assessments of rigor and relevance [3].…”