Vocabulary and time based bug‐assignment: A recommender system for open‐source projects

Sajedi‐Badashian, Ali; Stroulia, Eleni

doi:10.1002/spe.2830

Cited by 13 publications

(17 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is used to emphasize specific keywords and de-emphasize the common terms, and shown to work well on textual data. 16,17,41,44,45,99 Tf-idf gives an ''overlap score'' 79 to a document regarding the query. Having several documents (developer profiles), the document with the highest score is considered the document most similar (and most relevant as a response) to the given query (bug report).…”

Section: Experiments Setup and Data Setmentioning

confidence: 99%

Guidelines for evaluating bug‐assignment research

Sajedi‐Badashian

Stroulia

2020

J Software Evolu Process

Self Cite

View full text Add to dashboard Cite

Bug assignment is the task of ranking candidate developers in terms of their potential competence to fix a bug report. Numerous methods have been developed to address this task, relying on different methodological assumptions and demonstrating their effectiveness with a variety of empirical studies with numerous data sets and evaluation criteria. Despite the importance of the subject and the attention it has received from researchers, there is still no unanimity on how to validate and comparatively evaluate bug‐assignment methods and, often times, methods reported in the literature are not reproducible. In this paper, we first report on our systematic review of the broad bug‐assignment research field. Next, we focus on a few key empirical studies and review their choices with respect to three important experimental‐design parameters, namely, the evaluation metric(s) they report, their definition of who the real assignee is, and the community of developers they consider as candidate assignees. The substantial variability on these criteria led us to formulate a systematic experiment to explore the impact of these choices. We conducted our experiment on a comprehensive data set of bugs we collected from 13 long‐term open‐source projects, using a simple Tf‐IDf similarity metric. On the basis of our arguments and/or experiments, we provide useful guidelines for performing further bug‐assignment research. We conclude that mean average precision (MAP) is the most informative evaluation metric, the developer community should be defined as “all the project members,” and the real assignee should be defined as “any developer who worked toward fixing a bug.”

show abstract

Section: Experiments Setup and Data Setmentioning

confidence: 99%

Guidelines for evaluating bug‐assignment research

Sajedi‐Badashian

Stroulia

2020

J Software Evolu Process

Self Cite

View full text Add to dashboard Cite

show abstract

“…We use our previous method [17], VTBA, as the baseline method –since it was the latest text‐based IR model for BA and outperformed previous state‐of‐the‐art approaches. We further revise it to handle multiple sources of expertise and show that the improvement is non‐trivial.…”

Section: Background and Related Researchmentioning

confidence: 99%

“…In a recent research [17], we proposed a method called VTBA which is based on original TF‐IDF [18, 27–29] and showed that it outperforms many other recent approaches. It only considers the technical keywords (the ones that are a Stack Overflow tag) and disregards others.…”

Section: Bug Assignment Based On Multiple Sources Of Evidence Of Thmentioning

confidence: 99%

“…This enables us to rank the developers based on appropriateness for the given bug and do further processes regarding the top ones in the list. We defined the original VTBA score as follows [17]:

\begin{matrix} right leftthickmathspace.5em \\ score (q, d) & = {false\sum}_{i = 1}^{n} freq (t_{i}, q) \cdot w (t_{i}) \\ \times ({false\sum}_{j = 1}^{| SD |} t f (t_{i}, s {normald}_{j}) \cdot recency (s {normald}_{j})) \\ n : number of distinct terms in q \\ w (t_{i}) : term weight \\ SD : number of sub - documents in d . \end{matrix}

In the above equation, q is the query (bug description) and d is the document (developer's expertise).

normalS d_{j}

is the jth sub‐document of the document i (jth evidence of expertise of developer i ).…”

Section: Bug Assignment Based On Multiple Sources Of Evidence Of Thmentioning

confidence: 99%

“…In our previous research, we devised a new method, called vocabulary and time‐based bug‐assignment (VTBA) [17]. VTBA relies on two key intuitions.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Investigating the information value of different sources of evidence of developers’ expertise for bug assignment in open‐source projects

Sajedi‐Badashian

Stroulia

2020

IET softw.

Self Cite

View full text Add to dashboard Cite

Bug assignment (BA), the process of ranking developers according to their potential ability to fix a given bug, is an important software‐engineering task. BA usually requires the development of an expertise profile for each developer, and formulation of a similarity metric to estimate the relevance of developers to the bug. This needs us to answer the following question: ‘what is the information value of various contributions of developers in BA research?’ We address this question by making the following contributions. (i) We enhance the expertise metric of our prior work, vocabulary and time‐based BA, to consider information regarding various sources of expertise with different importance. We show that this can improve the effectiveness of bug‐assignment process. (ii) Using this ‘Multisource’ expertise metric, we investigate the information value of different pieces of information in open‐source repositories for BA. We show that in addition to bug‐fixing contributions, other technical and even social contributions of developers within the version‐control system are useful information for BA. (iii) We provide a curated, up‐to‐date data set including technical information of 13 popular open‐source projects in Github. To the best of our knowledge, this is the most comprehensive data set, currently available for bug‐assignment research.

show abstract

Dual analysis for helping developers to find collaborators based on co‐changed files: An empirical study

2023

View full text Add to dashboard Cite

Summary Software developers must collaborate at all stages of the software life‐cycle to create successful complex software systems. To enable this collaboration, social coding platforms, for example, GitHub, include an increasing number of tools to support collaboration. However, for large projects with hundreds of dynamic developers, such as several successful open–source projects, it can be complex to find developers with the same interest and familiarity and thus, gain suitable collaborations and new insights. In this context, resources and efforts may be wasted, discouraging many developers from contributing. Moreover, it can be costly to manage many contributions, which is another challenge for the maintainer who wants to take advantage of this small, timid, but valuable contribution made by a volunteer developer in a short time. In this context, this paper presents an empirical study aiming to evaluate two strategies to recommend collaborators based on co‐changed files. Inspired in the TF–IDF (Term Frequency–Inverse Document Frequency) weighting scheme established in the Information Retrieval field, these strategies first estimate the importance of relevant files modified by developers and use these estimates to represent each developer “profile”. As a second step, they estimate the similarity between developers using the Cosine metric, providing top‐ranked developers according to this measure as recommendations. We evaluated these strategies based on an extensive survey with 102 real–world developers. We observed that developers have interest and familiarity with the co‐changed files for all strategies evaluated. These considerations are of relevance because many opportunities for contributions to the project are linked to coding. Thus, theses results may indicate one less barrier for improving collaboration among developers. Overall, the strategies present an acceptance rate of up to 81%, contributing to the discovery of further collaborators.

show abstract

Vocabulary and time based bug‐assignment: A recommender system for open‐source projects

Cited by 13 publications

References 48 publications

Guidelines for evaluating bug‐assignment research

Guidelines for evaluating bug‐assignment research

Investigating the information value of different sources of evidence of developers’ expertise for bug assignment in open‐source projects

Dual analysis for helping developers to find collaborators based on co‐changed files: An empirical study

Contact Info

Product

Resources

About