Using Latent Dirichlet Allocation for automatic categorization of software

Tian, Kai; Revelle, Meghan; Poshyvanyk, Denys

doi:10.1109/msr.2009.5069496

Cited by 136 publications

(77 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…First, topic modeling using LDA has been applied to solve a wide range of problems in software engineering such as: statistical debugging (Andrzejewski et al 2007), mining business topics (Maskeri et al 2008), mining author-topic models (Linstead et al 2007b), software traceability (Asuncion et al 2010), software categorization (Tian et al 2009), bug localization (Lukins et al 2008) etc. In an earlier work we used LDA topic modeling to mine topics from large corpus of source code, and showed that topics that emerge often resemble widely known aspects or concerns in source code (Baldi et al 2008).…”

Section: Topic Modelingmentioning

confidence: 99%

Analyzing and mining a code search engine usage log

Bajracharya

Lopes

2010

Empir Software Eng

View full text Add to dashboard Cite

This paper presents an analysis of a year long usage log of Koders, the first commercially available Internet-Scale code search engine (http://www.koders.com). The usage log comprises about ten million activities from more than three million users. Analysis of the usage data shows that despite of attracting a large number of visitors, Koders has a very sparse usage and that it lacks regular usage from many of its users. When compared to Web search, search behavior in Koders showed many similar patterns. A topic modeling analysis of the usage data shows what topics users of Koders are looking for. Observations on the prevalence of these topics among the users, and observations on how search and download activities vary across topics, lead to the conclusion that users who find code search engines usable are those who already know to a high level of specificity what to look for. This paper also presents a general categorization of these topics that provides insights on the different ways code search engine users express their queries. It identifies various forms of queries in Koders's log and the kinds of results addressed by the queries. It also provides several suggestions for improvements in code search engines based on the analysis of usage, topics, and query forms. The work presented in this paper is the first of its kind that reveals several insights on the usage of an Internet-Scale code search engine.

show abstract

Section: Topic Modelingmentioning

confidence: 99%

Analyzing and mining a code search engine usage log

Bajracharya

Lopes

2010

Empir Software Eng

View full text Add to dashboard Cite

show abstract

“…LACT is another system that relies on information retrieval to categorize software (Tian et al 2009). LACT uses Latent Dirichlet Allocation (LDA) over the same dataset as Kawaguchi et al in order to infer topics to which applications belong.…”

Section: Related Workmentioning

confidence: 99%

On using machine learning to automatically classify software applications into domain categories

et al. 2012

Self Cite

View full text Add to dashboard Cite

Software repositories hold applications that are often categorized to improve the effectiveness of various maintenance tasks. Properly categorized applications allow stakeholders to identify requirements related to their applications and predict maintenance problems in software projects. Manual categorization is expensive, tedious, and laborious -this is why automatic categorization approaches are gaining widespread importance. Unfortunately, for different legal and organizational reasons, the applications' source code is often not available, thus making it difficult to automatically categorize these applications. In this paper, we propose a novel approach in which we use Application Programming Interface (API) calls from third-party libraries for automatic categorization of software applications that use these API calls. Our approach is general since it enables different categorization algorithms to be applied to repositories that contain both source code and bytecode of applications, since API calls can be extracted from both the source code and byte-code. We compare our approach to a state-of-the-art approach that uses machine learning algorithms for software categorization, and conduct experiments on two large Java repositories: an open-source repository containing 3,286 projects and a closed-source repository with 745 applications, where the source code was not available. Our contribution is twofold: we propose a new approach that makes it possible to categorize software projects without any source code using a small number of API calls as attributes, and furthermore we carried out a comprehensive empirical evaluation of automatic categorization approaches.

show abstract

“…In this section we provide details behind LDA followed by how RTM extends this model to capture links among documents. While LDA has been previously applied in the context of software engineering for measuring conceptual cohesion of classes [27], recovering traceability links [3,31], mining software repositories [4,30,36] and bug location [28], RTM has not been utilized for software measurement tasks before.…”

Section: Using Relational Topic Models For Coupling Measurement mentioning

confidence: 99%

“…Applying PCA to metrics data consist of the following steps: collecting the metrics data, identifying outliers, and performing PCA. We applied PCA in the similar manner as in our previous work [29,31,36], including procedures on identifying outliers and rotating principal components. Overall, by performing PCA we can identify groups of variables (i.e., coupling metrics), which are likely to measure the same underlying dimension (i.e., specific mechanism that defines coupling) of the object to be measured (i.e., coupling of classes).…”

Section: ) Principal Component Analysismentioning

confidence: 99%

“…RTM identifies latent topics associated with documents (e.g., source code elements) as well as links between documents in a large corpus of text. RTM extends on Latent Dirichlet Allocation (LDA) [7], which has been recently applied for extracting, representing and analyzing latent topics from the source code [4,27,28,30,31,36]. Our measure of coupling can be interpreted through deducing mixtures and relationships of latent topics implemented in software classes within the context of an entire system.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Using Relational Topic Models to capture coupling among classes in object-oriented software systems

Gethers

Poshyvanyk

2010

2010 IEEE International Conference on Software Maintenance

Self Cite

View full text Add to dashboard Cite

-Coupling metrics capture the degree of interaction and relationships among source code elements in software systems. A vast majority of existing coupling metrics rely on structural information, which captures interactions such as usage relations between classes and methods or execute after associations. However, these metrics lack the ability to identify conceptual dependencies, which, for instance, specify underlying relationships encoded by developers in identifiers and comments of source code classes. We propose a new coupling metric for object-oriented software systems, namely Relational Topic based Coupling (RTC) of classes, which uses Relational Topic Models (RTM), generative probabilistic model, to capture latent topics in source code classes and relationships among them. A case study on thirteen open source software systems is performed to compare the new measure with existing structural and conceptual coupling metrics. The case study demonstrates that proposed metric not only captures new dimensions of coupling, which are not covered by the existing coupling metrics, but also can be used to effectively support impact analysis.

show abstract

Using Latent Dirichlet Allocation for automatic categorization of software

Abstract: Abstract

Cited by 136 publications

References 19 publications

Analyzing and mining a code search engine usage log

Analyzing and mining a code search engine usage log

On using machine learning to automatically classify software applications into domain categories

Using Relational Topic Models to capture coupling among classes in object-oriented software systems

Contact Info

Product

Resources

About