Regression Testing Goals - View of Practitioners and Researchers

Abstract

Context: Regression testing is a well-researched area. However, the majority regression testing techniques proposed by the researchers are not getting the attention of the practitioners. Communication gaps between industry and academia, and disparity in the regression testing goals are the main reasons. Close collaboration can help in bridging the communication gaps and resolving the disparities. Objective: The study aims at exploring the views of academics and practitioners about the goals of regression testing. The purpose is to investigate the commonalities and differences in their viewpoints and defining some common goals for the success of regression testing. Method: We conducted a focus group study, with 7 testing experts from industry and academia. 4 testing practitioners from 2 companies and 3 researchers from 2 universities participated in the study. We followed GQM approach, to elicit the regression testing goals, information needs, and measures. Results: 43 regression testing goals were identified by the participants, which were reduced to 10 on the basis of similarity among the identified goals. Later during the priority assignment process, 5 goals were discarded, because the priority assigned to these goals was very low. Participants identified 47 information needs/questions required to evaluate the success of regression testing with reference to goal G5 (confidence). Which were then reduced to 10 on the basis of similarity. Finally, we identified measures to gauge those information needs/questions, which were corresponding to the goal (G5). Conclusions: We observed that participation level of practitioners and researchers during the elicitation of goals and questions was same. We found a certain level of agreement between the participants regarding the regression testing definitions and goals. But there was some level of disagreement regarding the priorities of the goals. We also identified the need to implement a regression testing evaluation framework in the participating companies.

Regression testing for large-scale embedded software development – Exploring the state of practice

“…There is only little work on the perspectives of testing practitioners in industry on regression testing. We found only 13 articles [4,5,[13][14][15][16][17][18][19][20][21][22][23] related to our work, which we discuss further below. Out of these 13 studies, eight deal specifically with regression testing, whereas five have a broader focus.…”

“…Minhas et al [16] conducted a focus group based study to investigate the views of industry practitioners and software engineering researchers concerning regression testing. The authors explore the perceptions of both communities about the scope of regression testing.…”

“…For the regression testing challenges, we also utilized the study by Brahneborg et al [15]. Regarding RT goals we extracted the information from [16,25]. We have listed the key findings of these studies in Table 2.…”

“…Increasing confidence regarding the system reliability (confidence) [16,25] LG4 Identifying high risk (critical) faults [25] LG5…”

On the search for industry-relevant regression testing research

“…As for testing, in general, the value of a regression testing technique could be measured either in terms of its ability to increase confidence in testing or in terms of its ability to improve fault detection with limited time and resources. Those high-level goals are shared by researchers and practitioners but with some variations when it comes to details and priorities (Minhas et al 2017). The recommendations given here are based on our comparison of what we found in the literature and the feedback from our industry partners.…”