Software traceability with topic modeling

Abstract

Software traceability is a fundamentally important task in software engineering. The need for automated traceability increases as projects become more complex and as the number of artifacts increases. We propose an automated technique that combines traceability with a machine learning technique known as topic modeling. Our approach automatically records traceability links during the software development process and learns a probabilistic topic model over artifacts. The learned model allows for the semantic categorization of artifacts and the topical visualization of the software system. To test our approach, we have implemented several tools: an artifact search tool combining keyword-based search and topic modeling, a recording tool that performs prospective traceability, and a visualization tool that allows one to navigate the software architecture and view semantic topics associated with relevant artifacts and architectural components. We apply our approach to several data sets and discuss how topic modeling enhances software traceability, and vice versa.

Analyzing and mining a code search engine usage log

“…First, topic modeling using LDA has been applied to solve a wide range of problems in software engineering such as: statistical debugging (Andrzejewski et al 2007), mining business topics (Maskeri et al 2008), mining author-topic models (Linstead et al 2007b), software traceability (Asuncion et al 2010), software categorization (Tian et al 2009), bug localization (Lukins et al 2008) etc. In an earlier work we used LDA topic modeling to mine topics from large corpus of source code, and showed that topics that emerge often resemble widely known aspects or concerns in source code (Baldi et al 2008).…”

Application of reinforcement learning to requirements engineering: requirements tracing

“…The Value Iteration algorithm is an iterative backup operation. The algorithm combines an immediate policy improvement for the current state and the values of states reachable from the current state in the following form: (4) where P a ss and R a ss` bear the same meaning as defined in Equation 2. The value of state s is maximized across all actions a available at s. The pseudo code for the Value Iteration algorithm is shown below:…”

Using Relational Topic Models to capture coupling among classes in object-oriented software systems

“…In this section we provide details behind LDA followed by how RTM extends this model to capture links among documents. While LDA has been previously applied in the context of software engineering for measuring conceptual cohesion of classes [27], recovering traceability links [3,31], mining software repositories [4,30,36] and bug location [28], RTM has not been utilized for software measurement tasks before.…”

“…The description of the study follows the Goal-Question-Metrics paradigm outlined by Basili et al [5]. All the data used and generated in this section has been posted online to ensure reproducibility of our results 3 .…”