Not all bugs are the same: Understanding, characterizing, and classifying bug types

Catolino, Gemma; Palomba, Fabio; Zaidman, Andy; Ferrucci, Filomena

doi:10.1016/j.jss.2019.03.002

Cited by 91 publications

(51 citation statements)

References 81 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The complex phenomenon of bug introduction has been studied before. Previous studies have helped researchers to understand that fixing bugs consist of determining why software behaves erroneously (Zeller 2009;Beller et al 2018), that bugs can have different root causes (Li et al 2006;Catolino et al 2019), and that bugs can be introduced in a version of the software system but were not found until much later (Chen et al 2014). However, the state-of-the-art lacks a better understanding of the origin of bugs.…”

Section: Towards a Better Understanding Of Bug Introductionmentioning

confidence: 99%

How bugs are born: a model to identify how bugs are introduced in software components

et al. 2020

Self Cite

View full text Add to dashboard Cite

When identifying the origin of software bugs, many studies assume that "a bug was introduced by the lines of code that were modified to fix it". However, this assumption does not always hold and at least in some cases, these modified lines are not responsible for introducing the bug. For example, when the bug was caused by a change in an external API. The lack of empirical evidence makes it impossible to assess how important these cases are and therefore, to which extent the assumption is valid. To advance in this direction, and better understand how bugs "are born", we propose a model for defining criteria to identify the first snapshot of an evolving software system that exhibits a bug. This model, based on the perfect test idea, decides whether a bug is observed after a change to the software. Furthermore, we studied the model's criteria by carefully analyzing how 116 bugs were introduced in two different open source software projects. The manual analysis helped classify the root cause of those bugs and created manually curated datasets with bug-introducing changes and with bugs that were not introduced by any change in the source code. Finally, we used these datasets to evaluate the performance of four existing SZZ-based algorithms for detecting bug-introducing changes. We found that SZZ-based algorithms are not very accurate, especially when multiple commits are found; the F-Score varies from 0.44 to 0.77, while the percentage of true positives does not exceed 63%. Our results show empirical evidence that the prevalent assumption, "a bug was introduced by the lines of code that were modified to fix it", is just one case of how bugs are introduced in a software system. Finding what introduced a bug is not trivial: bugs can be introduced by the developers and be in the code, or be created irrespective of the code. Thus, further research towards a better understanding of the origin of bugs in software projects could help to improve design integration tests and to design other procedures to make software development more robust. Keywords Bug origins • Bug-introducing changes • First-failing change • SZZ algorithm • Extrinsic bugs • Intrinsic bugs Communicated by: Per Runeson

show abstract

Section: Towards a Better Understanding Of Bug Introductionmentioning

confidence: 99%

How bugs are born: a model to identify how bugs are introduced in software components

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…They suggested that for software performance, factors different from the time were underrepresented, such as memory usage and throughput. Catolino et al [22] analyzed bug reports of different popular projects with the aim of building a taxonomy of the types of bugs. This work provided a taxonomy for both functional and non-functional bugs, categorizing most of the performance bugs in the category "performance issue".…”

Section: Related Workmentioning

confidence: 99%

“…Functional bugs and their classification have been extensively studied since many years ago [15]. However, existing performance-related classifications are either too general [16], target specific scopes (e.g., cloud computing systems or Android apps) [17]- [21] or address both functional and performance bugs [22]. This makes even more difficult for researchers to find subject bugs for their studies, and as a result, they are finally advocated to perform expensive and often unsuccessful searches in software repositories from scratch.…”

Section: Introductionmentioning

confidence: 99%

TANDEM: A Taxonomy and a Dataset of Real-World Performance Bugs

et al. 2020

View full text Add to dashboard Cite

The detection of performance bugs, like those causing an unexpected execution time, has gained much attention in the last years due to their potential impact in safety-critical and resourceconstrained applications. Much effort has been put on trying to understand the nature of performance bugs in different domains as a starting point for the development of effective testing techniques. However, the lack of a widely accepted classification scheme of performance faults and, more importantly, the lack of well-documented and understandable datasets makes it difficult to draw rigorous and verifiable conclusions widely accepted by the community. In this paper, we present TANDEM, a dual contribution related to real-world performance bugs. Firstly, we propose a taxonomy of performance bugs based on a thorough systematic review of the related literature, divided into three main categories: effects, causes and contexts of bugs. Secondly, we provide a complete collection of fully documented real-world performance bugs. Together, these contributions pave the way for the development of stronger and reproducible research results on performance testing. INDEX TERMS performance bugs, performance testing, dataset, taxonomy.

show abstract

“…A análise foi realizada utilizando o estudo dos defeitos reportados por módulos, prioridade e resolução dos defeitos através da ferramenta que utiliza aprendizado de máquina para análise dos textos. O estudo de [Catolino et al 2019] procurou entender, caracterizar e classificar defeitos reportados em issues dos projetos Mozilla, Apache e Eclipse. No estudo foram analisados 1.280 bugs que possibilitou a elaboração de uma taxonomia de tipos de bugs reportados.…”

Section: Trabalhos Relacionadosunclassified

Avaliação empírica de termos técnicos em issues de projetos opensource

Júnior

Machado

2020

REIC

View full text Add to dashboard Cite

A demanda crescente por sistemas de software robustos e confiáveis tem levado a comunidade científica e a indústria de software a investigar aspectos relacionados aos defeitos reportados em sistemas de rastreamento de defeitos. O objetivo comum é compreender as causas dos defeitos, e assim estabelecer estratégias para prevenir sua ocorrência e propagação, reduzindo possíveis danos. Os defeitos reportados por usuários através de issues em rastreadores de defeitos revelam informações importantes para os desenvolvedores. A presente investigação realizou uma análise de dados provenientes de 510.212 issues de 85 repositórios de projetos de código-fonte aberto, disponíveis no Github. Com o auxílio de uma lista de 608 termos técnicos selecionados manualmente, foi analisada a frequência na utilização dos termos pelos contribuidores desses repositórios. O estudo encontrou o emprego dos termos nos diferentes campos de uma issue, bem como a possibilidade de identificação dos termos mais utilizados nestes repositórios. Os resultados deste estudo apontam para ganhos potenciais na compreensão do ambiente de falhas por apresentar termos que formam indicativos de defeitos em estruturas fundamentais do repositório.

show abstract

Not all bugs are the same: Understanding, characterizing, and classifying bug types

Cited by 91 publications

References 81 publications

How bugs are born: a model to identify how bugs are introduced in software components

How bugs are born: a model to identify how bugs are introduced in software components

TANDEM: A Taxonomy and a Dataset of Real-World Performance Bugs

Avaliação empírica de termos técnicos em issues de projetos opensource

Contact Info

Product

Resources

About