Factors Impacting the Inputs of Traceability Recovery Approaches

Abstract

In requirement engineering, researchers have proposed various tractability recovery approaches. To the best of our knowledge, all traceability recovery approaches have low precision and recall. Our main claim in this chapter is that there exist factors that impact the traceability approaches' inputs, in particular source document, target document, and experts' opinion, that cause low precision and recall. In this chapter, we pursue four objectives: first, to identify and document factors that impact traceability recovery approaches' inputs; second, to identify metrics/tools to measure/improve the quality of the inputs with respect to the identified factors, third, to provide precautions to control these factors, and, fourth, to empirically prove and quantify the effect of one of these factors-expert's programming knowledge-on the traceability recovery approaches' inputs. To achieve the first two objectives, we perform an incremental literature review of traceability recovery approaches and identify and document three key inputs and the seven factors impacting these inputs, out of 12 identified factors. We analyse the reported results in literature for the identified factors to address our third objective. We conduct an empirical study to assess the impact of expert's programming knowledge, to address our fourth objective. We use the effort, number of correct answers, and time to measure the effect of expert's programming knowledge on traceability recovery. We conclude that, in the literature, seven factors impacting the inputs of traceability recovery approaches have been identified, documented, and reported along with related metrics/tools and precautions. We suggest that practitioners should be wary of these seven factors and researchers should focus on the five others to improve traceability recovery approaches.

Trustrace: Mining Software Repositories to Improve the Accuracy of Requirement Traceability Links

“…Figure 4 shows that Trustrace has a better accuracy, on average, when compared to JSM and VSM for jEdit and Rhino but less for Pooka and SIP. We explain the differences in improvements by the many other factors that can impact the accuracy of traceability-recovery approaches, as discussed elsewhere [10]. One of these factors is quality of source code identifiers.…”

“…Low similarity values among different documents would result into low precision and recall values for the traceability links [10]. Figure 6 shows that JSM provides better similarity values between documents than VSM.…”

“…A second factor from Ali et al [10] is the knowledge of the developers who wrote the requirements. In our empirical evaluation, we recovered requirements for Pooka and SIP using Prereqir [7].…”

“…Some researchers [8], [9] performed several experiments using various weights to provide a range of weights that work well with their approaches/datasets. However, we cannot generalise such kinds of weights, because every dataset is unique [10]. In addition, a range of weights is only a start to recover traceability links and, without an oracle, it is impossible to identify what weights and-or ranges of weights are suitable.…”

“…Requirements and source code consequently diverge from each other, which decreases their textual similarity. Thus, IR techniques (or any combination thereof [8], [9]) typically produce links with low precision and-or recall because, due to their very nature, they depend on the textual similarity between requirements and source code [10]. Yet, while developers may not evolve requirements in synchronisation with source code, they frequently update other sources of information, including CVS/SVN repositories, bug-tracking systems, mailing lists, forums, and blogs.…”