Exploring the benefits of utilizing conceptual information in test-to-code traceability

Abstract

Striving for reliability of software systems often results in immense numbers of tests. Due to the lack of a generally used annotation, finding the parts of code these tests were meant to assess can be a demanding task. This is a valid problem of software engineering called test-to-code traceability. Recent research on the subject has attempted to cope with this problem applying various approaches and their combinations, achieving profound results. These approaches have involved the use of naming conventions during development processes and also have utilized various information retrieval (IR) methods often referred to as conceptual information. In this work we investigate the benefits of textual information located in software code and its value for aiding traceability. We evaluated the capabilities of the natural language processing technique called Latent Semantic Indexing (LSI) in the view of the results of the naming conventions technique on five real, medium sized software systems. Although LSI is already used for this purpose, we extend the viewpoint of one-to-one traceability approach to the more versatile view of LSI as a recommendation system. We found that considering the top 5 elements in the ranked list increases the results by 30% on average and makes LSI a viable alternative in projects where naming conventions are not followed systematically. CCS CONCEPTS• Computing methodologies → Natural language processing; • Software and its engineering → Traceability;

xNose: A Test Smell Detector for C#

“…For instance, for identifying Lazy Test [46], we need to know the production files and classes that correspond to it. Numerous recent studies have examined the topic of test-tocode traceability [28], [29], [25], and a wide range of approaches have been proposed. Nonetheless, establishing a dependable 1-to-1 relationship between a production method and a test method within the context of static analysis presents a challenge [29].…”

Applications of Natural Language Processing in Software Traceability: A Systematic Mapping Study

“…Lack of generally used annotation of artifacts (Kicsi et al, 2018) and imperfectly appropriate naming (Csuvik et al, 2019b) typically lead to inaccurate links. The added challenge of artifacts, such as non-functional requirements (Mahmoud and Williams, 2016), hinders traceability efforts due to the lack of homogeneity in syntax representation: natural language pertaining to nonfunctional requirements is less explicit in tracing links.…”

Utilization of Underlying Semantic Information in Textual Data

“…Thus, our results were evaluated on eight real programs and also using a variety of source code representations and settings. Our work aimed to show that LSI itself performs better than it had been previously perceived by the research community [75], investigated the question of source code representations in the task, and also found that Doc2Vec can significantly outperform LSI [31] while a suitable combination of the textual similarity techniques could provide even better results [30].…”

“…While test-to-code traceability is not the most widespread topic amongst recovery tasks, several well-known approaches aim to cope with this problem. Still, as yet, none of them has provided a perfect solution for the problem [61,132,75,126,31,30]. The current state-of-the-art techniques [125] rely on a combination of diverse methods -i.e.…”

“…The use of recommendation systems is widespread in the field of software engineering [80,128,129]. Presenting a prioritized list of most likely solutions seems to be a more resilient approach even in traceability research [75,31].…”