Statistical Debugging Using Latent Topic Models

Abstract

Abstract. Statistical debugging uses machine learning to model program failures and help identify root causes of bugs. We approach this task using a novel Delta-Latent-Dirichlet-Allocation model. We model execution traces attributed to failed runs of a program as being generated by two types of latent topics: normal usage topics and bug topics. Execution traces attributed to successful runs of the same program, however, are modeled by usage topics only. Joint modeling of both kinds of traces allows us to identify weak bug topics that would otherwise remain undetected. We perform model inference with collapsed Gibbs sampling. In quantitative evaluations on four real programs, our model produces bug topics highly correlated to the true bugs, as measured by the Rand index. Qualitative evaluation by domain experts suggests that our model outperforms existing statistical methods for bug cause identification, and may help support other software tasks not addressed by earlier models.

Analyzing and mining a code search engine usage log

“…First, topic modeling using LDA has been applied to solve a wide range of problems in software engineering such as: statistical debugging (Andrzejewski et al 2007), mining business topics (Maskeri et al 2008), mining author-topic models (Linstead et al 2007b), software traceability (Asuncion et al 2010), software categorization (Tian et al 2009), bug localization (Lukins et al 2008) etc. In an earlier work we used LDA topic modeling to mine topics from large corpus of source code, and showed that topics that emerge often resemble widely known aspects or concerns in source code (Baldi et al 2008).…”

Learning Rare Behaviours

“…Our algorithm is able to learn joint models of both typical and rare behaviours even if they co-exist. MC-∆LDA is a generalisation and completion of the ∆LDA model proposed in [12]. ∆LDA was used for understanding code bugs in computer programs, but without an inference framework for the labels of unseen documents, ∆LDA cannot be used for classification.…”

Cooperative Bug Isolation

“…We follow the sitesand-predicates approach commonly used in prior work [1,17,[26][27][28]45]. An instrumentation site is a single program location at which the state of the running program will be inspected.…”

“…Statistical debugging techniques monitor run-time behavior to identify causes of crashes in end-user executions. Lightweight instrumentation [26] allows non-intrusive post-deployment monitoring, while statistical models [1,17,18,[26][27][28]45] identify profiled events that strongly predict crashes or other failures. Yet most programs mostly work: nearly all code in any given application is not relevant for any given bug.…”