Finding latent performance bugs in systems implementations

Killian, Charles; Nagaraj, Karthik; Pervez, Salman; Braud, Ryan; Anderson, James W.; Jhala, Ranjit

doi:10.1145/1882291.1882297

Cited by 49 publications

(24 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Following the convention of developers and researchers on this topic [5,26,40,50], we refer to performance bugs as software Performance bugs exist widely in released software. For example, Mozilla developers have fixed 5-60 performance bugs reported by users every month over the past 10 years.…”

Section: Motivationmentioning

confidence: 99%

Understanding and detecting real-world performance bugs

Jin

Song

Shi

et al. 2012

Proceedings of the 33rd ACM SIGPLAN Conference on Programming Language Design and Implementation

222

106

View full text Add to dashboard Cite

Developers frequently use inefficient code sequences that could be fixed by simple patches. These inefficient code sequences can cause significant performance degradation and resource waste, referred to as performance bugs. Meager increases in single threaded performance in the multi-core era and increasing emphasis on energy efficiency call for more effort in tackling performance bugs.This paper conducts a comprehensive study of 109 real-world performance bugs that are randomly sampled from five representative software suites (Apache, Chrome, GCC, Mozilla, and MySQL). The findings of this study provide guidance for future work to avoid, expose, detect, and fix performance bugs.Guided by our characteristics study, efficiency rules are extracted from 25 patches and are used to detect performance bugs. 332 previously unknown performance problems are found in the latest versions of MySQL, Apache, and Mozilla applications, including 219 performance problems found by applying rules across applications.

show abstract

Section: Motivationmentioning

confidence: 99%

Understanding and detecting real-world performance bugs

Jin

Song

Shi

et al. 2012

Proceedings of the 33rd ACM SIGPLAN Conference on Programming Language Design and Implementation

222

106

View full text Add to dashboard Cite

show abstract

“…Previous studies have shown that this type of performance-related software defects 1 widely exist in real-world [10,21,24,35,41]. They are difficult for developers to avoid due to the lack of performance documentation of APIs and the quickly changing workload of modern software [21].…”

Section: Motivationmentioning

confidence: 99%

Statistical debugging for real-world performance problems

Song

2014

Proceedings of the 2014 ACM International Conference on Object Oriented Programming Systems Languages &Amp; Applications

View full text Add to dashboard Cite

Design and implementation defects that lead to inefficient computation widely exist in software. These defects are difficult to avoid and discover. They lead to severe performance degradation and energy waste during production runs, and are becoming increasingly critical with the meager increase of single-core hardware performance and the increasing concerns about energy constraints. Effective tools that diagnose performance problems and point out the inefficiency root cause are sorely needed.The state of the art of performance diagnosis is preliminary. Profiling can identify the functions that consume the most computation resources, but can neither identify the ones that waste the most resources nor explain why. Performance-bug detectors can identify specific type of inefficient computation, but are not suited for diagnosing general performance problems. Effective failure diagnosis techniques, such as statistical debugging, have been proposed for functional bugs. However, whether they work for performance problems is still an open question.In this paper, we first conduct an empirical study to understand how performance problems are observed and reported by real-world users. Our study shows that statistical debugging is a natural fit for diagnosing performance problems, which are often observed through comparison-based approaches and reported together with both good and bad inputs. We then thoroughly investigate different design points in statistical debugging, including three different predicates and two different types of statistical models, to understand which design point works the best for performance diagnosis. Finally, we study how some unique nature of performance bugs allows sampling techniques to lower the over- * Shan is now with University of Chicago.

show abstract

“…Previous study has shown that this type of performance-related software defects 1 widely exist in real- 1 We will refer to these defects as performance bugs or performance problems interchangeably following previous work in this area [18,21,30]. world [9,18,21,28,34].…”

Section: Motivationmentioning

confidence: 99%

Statistical debugging for real-world performance problems

SongLinhai

2014

SIGPLAN Not.

View full text Add to dashboard Cite

Design and implementation defects that lead to inefficient computation widely exist in software. These defects are difficult to avoid and discover. They lead to severe performance degradation and energy waste during production runs, and are becoming increasingly critical with the meager increase of single-core hardware performance and the increasing concerns about energy constraints. Effective tools that diagnose performance problems and point out the inefficiency root cause is sorely needed.The state of the art of performance diagnosis is preliminary. Profiling can tell where computation resources are spent, but not where and why the resources are wasted. Performance-bug detectors can identify specific type of inefficient computation, but are not suited for diagnosing general performance problems. Effective failure diagnosis techniques, such as statistical debugging, have been proposed for functional bugs. However, whether they work for performance problems is still an open question.In this paper, we first conduct an empirical study to understand how performance problems are observed and reported by real-world users. Our study shows that statistical debugging is a natural fit for diagnosing performance problems, which are often observed through comparison-based approaches and reported together with both good and bad inputs. We then thoroughly investigate different design points in statistical debugging, including three different predicates and two different types of statistical models, to understand which design point works the best for performance diagnosis. Finally, we study how some unique nature of performance bugs allows sampling techniques to lower the overhead of run-time performance diagnosis without extending the diagnosis latency.

show abstract

Finding latent performance bugs in systems implementations

Cited by 49 publications

References 22 publications

Understanding and detecting real-world performance bugs

Understanding and detecting real-world performance bugs

Statistical debugging for real-world performance problems

Statistical debugging for real-world performance problems

Contact Info

Product

Resources

About