Proximity-based traceability: An empirical validation using ranked retrieval and set-based measures

Kong, Wei-Keat; Hayes, Jane Huffman

doi:10.1109/empire.2011.6046255

Cited by 9 publications

(6 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Applications of IR to TLR are the most related to this work, and began with probabilistic [36], [38] and vector space models [39] to retrieve links between source code and documentation, as well as source code and requirements. Additionally, other IR approaches such as Latent Semantic Analysis (LSA) [5], probabilistic LSA (pLSA) [6], Jensen-Shannon (JS) [6], Latent Dirichlet Allocation (LDA) [7], and proximity-based VSM [8] have also been applied directly to the TLR task. Oliveto et al performed an empirical study of IR methods for TLR [17] comparing VSM, LSA, LDA, and JS via Principal Component Analysis, showing that VSM, LSA, and JS capture similar information, while information captured by LDA is unique.…”

Section: Related Workmentioning

confidence: 99%

“…Establishing traceability links between the artifacts of a system is extremely arduous and error-prone when performed manually. This has led to a large body of research proposing techniques that aid developers with this task [4], [5], [6], [7], [8]. However, establishing traceability links at one point in the lifetime of a software system is only part of the struggle.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Automatic Traceability Maintenance via Machine Learning Classification

Mills

Escobar-Avila

Haiduc

2018

2018 IEEE International Conference on Software Maintenance and Evolution (ICSME)

View full text Add to dashboard Cite

Previous studies have shown that software traceability, the ability to link together related artifacts from different sources within a project (e.g., source code, use cases, documentation, etc.), improves project outcomes by assisting developers and other stakeholders with common tasks such as impact analysis, concept location, etc. Establishing traceability links in a software system is an important and costly task, but only half the struggle. As the project undergoes maintenance and evolution, new artifacts are added and existing ones are changed, resulting in outdated traceability information. Therefore, specific steps need to be taken to make sure that traceability links are maintained in tandem with the rest of the project.In this paper we address this problem and propose a novel approach called TRAIL for maintaining traceability information in a system. The novelty of TRAIL stands in the fact that it leverages previously captured knowledge about project traceability to train a machine learning classifier which can then be used to derive new traceability links and update existing ones. We evaluated TRAIL on 11 commonly used traceability datasets from six software systems and compared it to seven popular Information Retrieval (IR) techniques including the most common approaches used in previous work. The results indicate that TRAIL outperforms all IR approaches in terms of precision, recall, and F-score.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Automatic Traceability Maintenance via Machine Learning Classification

Mills

Escobar-Avila

Haiduc

2018

2018 IEEE International Conference on Software Maintenance and Evolution (ICSME)

View full text Add to dashboard Cite

show abstract

“…Such activities tend to be costly to implement and are therefore perceived as financially nonviable by many companies [8], [15]. To address this problem, many efforts [16], [17], [18], [19], [20], [21], [22] have been devoted to semi-automatic or fullyautomatic trace link creation. However, the precision and recall of generated trace links is still at a low level of accuracy, such that the trace link creation throughout the entire systems development process, remains a challenging issue [23].…”

Section: Research Questionsmentioning

confidence: 99%

Requirements development and management of embedded real-time systems

Zhou

2014

2014 IEEE 22nd International Requirements Engineering Conference (RE)

View full text Add to dashboard Cite

It is well recognized that most of the anomalies, discovered in the development of embedded real-time systems, belong to requirement and specification phases. To ease the situation, many efforts have been investigated into the area. For requirements development, especially requirements validation and verification, model-driven architecture techniques can be considered as a cost-efficient solution. In order to utilize such advantages, the design of the proposed system is often specified in terms of analyzable models at the certain level of abstraction. Further, different levels of requirements are translated into verifiable queries and fed into the models to be either validated or verified. For requirements management, requirements traceability provides critical support for performing change impact analysis, risk analysis, regression testing, etc. In this thesis, we cover several topics about requirements validation, requirements verification, and requirements traceability. In particular, the technical contributions are three-fold: 1) we propose an approach to requirements validation by using the extended Timed Abstract State Machine (TASM) language with newly defined TASM constructs and, 2) we present a simulation-based method which is powered up by statistical techniques to conduct requirements verification, working with industrial applications and, 3) we introduce an improved VSM-based requirements traceability recovery approach using a novel context analysis. Further, we have demonstrated the applicability of our contributions in real world usage through various case studies.

show abstract

“…The study of automated tracing methods has resulted in much progress toward the automation of candidate trace link generation. Techniques using latent semantic analysis [11], key phrases [14], unsupervised learning [15], and term proximity [13] exploit the structural relationship between words in a document. The use of thesauri [10,16] and web queries [17] supplement trace link generation with external information to improve weak links.…”

Section: Background and Related Workmentioning

confidence: 99%

“…Recall and precision are measures frequently used to evaluate the accuracy of a TM from a researcher's perspective [8,9,10,11,12], while measures such as lag, selectivity, and mean average precision have been used to evaluate the quality of a TM from an analyst's perspective [5,13]. In general, automated methods return candidate TMs ("candidate" until a human analyst vets them) with high recall and low precision.…”

Section: Introductionmentioning

confidence: 99%

Process improvement for traceability: A study of human fallibility

Kong

Hayes

Dekhtyar

et al. 2012

2012 20th IEEE International Requirements Engineering Conference (RE)

Self Cite

View full text Add to dashboard Cite

Abstract-Human analysts working with results from automated traceability tools often make incorrect decisions that lead to lower quality final trace matrices. As the human must vet the results of trace tools for mission-and safetycritical systems, the hopes of developing expedient and accurate tracing procedures lies in understanding how analysts work with trace matrices. This paper describes a study to understand when and why humans make correct and incorrect decisions during tracing tasks through logs of analyst actions. In addition to the traditional measures of recall and precision to describe the accuracy of the results, we introduce and study new measures that focus on analyst work quality: potential recall, sensitivity, and effort distribution. We use these measures to visualize analyst progress towards the final trace matrix, identifying factors that may influence their performance and determining how actual tracing strategies, derived from analyst logs, affect results.

show abstract

Proximity-based traceability: An empirical validation using ranked retrieval and set-based measures

Cited by 9 publications

References 31 publications

Automatic Traceability Maintenance via Machine Learning Classification

Automatic Traceability Maintenance via Machine Learning Classification

Requirements development and management of embedded real-time systems

Process improvement for traceability: A study of human fallibility

Contact Info

Product

Resources

About