Detection and Diagnosis of Recurrent Faults in Software Systems by Invariant Analysis

2010 International Conference on Network and Service Management

Ward

2010

Self Cite

In recent years autonomic computing, specifically autonomic data centre management has gained significant attention. Human intervention be minimized to reduce the operating costs of business applications. In this paper we focus our attention to the self-repair dimension and present a flexible probabilistic framework to develop agents for self-repair in the context of business-information-system components. Our framework seeks to pick the optimal sequence of repair actions given only imperfect information about the experienced fault. In contrast to existing recovery-oriented approaches, our model explicitly considers fault prevalence, symptoms of recurrent failures, and inclusive repair actions.We evaluate our proposal using discrete event simulation. Our evaluation shows that an optimal repair policy can be computed from a brief specification of repair actions. Even in the context of very unreliable error detection our controller is able to estimate the current state of the monitored system and recover from failure.

Section: Incorporating Symptoms Of Known Faultsmentioning

confidence: 99%

“…The diagnostic approaches differ by monitoring data and overhead. To localize abnormal components from application traces we refer the reader to [16], for identifying recurrent faults from log-files to [13], [14], and finally for localizing faulty components the reader is referred to [15], [17]- [19].…”

Section: Incorporating Symptoms Of Known Faultsmentioning

confidence: 99%

An extensible framework for repair-driven monitoring

2010 International Conference on Network and Service Management

Ward

2010

Self Cite

“…By merging those samples into one class and then training a Naïve Bayes classifier, one can improve the performance of the Bayesian Classifier. Those particular faults then need to be diagnosed by other means [12,13,21].…”

Section: Dealing With Ambiguous Fault Manifestationsmentioning

confidence: 99%

“…Once a failure is validated (e.g., as described in previous work [12,13,21]), we seek to identify recurrent faults that underlie the observed failure. If the fault that triggered a failure has been seen before, we may be able to fast-track resolution by retrieving information about past actions to restore the failed component.…”

Section: Learning Fault Manifestationsmentioning

confidence: 99%

Diagnosis of recurrent faults using log files

Munawar

Proceedings of the 2009 Conference of the Center for Advanced Studies on Collaborative Research - CASCON '09

et al. 2009

Self Cite

Enterprise software systems (ESS) are becoming larger and increasingly complex. Failure in business-critical systems is expensive, leading to consequences such as loss of critical data, loss of sales, customer dissatisfaction, even law suits. Therefore, detecting failures and diagnosing their root-cause in a timely manner is essential. Many studies suggest that a large fraction of failures encountered in practice are recurrent (i.e., they have been seen before). Fast and accurate detection of these failures can accelerate problem determination, and thereby improve system reliability. To this effect, we explore machine learning techniques, including the Naïve Bayes classifier, partially-supervised learning, and decision trees (using C4.5), to automatically recognize symptoms of recurrent faults and to derive detection rules from samples of log data. This work focuses on log files, since they are readily available and they do not put any additional computational burden on the component generating the data.The methods explored in this work can aid the development of tools to assist support personnel in problem determination tasks. Instead of requiring the operators to manually define patterns for identifying recurrent problems, such tools can be trained using prior, solved and unsolved cases from exist-

“…Typically, supervised learning is used such that any recurrent faults can be identified quickly if they were to occur again. Such methods [10], [11], [12] use either Bayesian or neural networks to learn fault symptoms from labeled data. While such efforts significantly improve diagnosis, they suffer from the important drawback that prior fault knowledge is required to diagnose a fault successfully.…”

Section: Fault Diagnosismentioning

confidence: 99%

Efficient Fault Detection and Diagnosis in Complex Software Systems with Information-Theoretic Monitoring

Munawar

IEEE Trans. Dependable and Secure Comput.

et al. 2011

Self Cite

Management metrics of complex software systems exhibit stable correlations which can enable fault detection and diagnosis. Current approaches use specific analytic forms, typically linear, for modeling correlations. In practice, more complex nonlinear relationships exist between metrics. Moreover, most intermetric correlations form clusters rather than simple pairwise correlations. These clusters provide additional information and offer the possibility for optimization. In this paper, we address these issues by using Normalized Mutual Information (NMI) as a similarity measure to identify clusters of correlated metrics, without assuming any specific form for the metric relationships. We show how to apply the Wilcoxon Rank-Sum test on the entropy measures to detect errors in the system. We also present three diagnosis algorithms to locate faulty components: RatioScore, based on the Jaccard coefficient, SigScore, which incorporates knowledge of component dependencies, and BayesianScore, which uses Bayesian inference to assign a fault probability to each component. We evaluate our approach in the context of a complex enterprise application, and show that 1) stable, nonlinear correlations exist and can be captured with our approach; 2) we can detect a large fraction of faults with a low false positive rate (we detect up to 18 of the 22 faults we injected); and 3) we improve the diagnosis with our new diagnosis algorithms.