Adaptive Algorithms for Diagnosing Large-Scale Failures in Computer Networks

Tati, Srikar; Ko, Bong Jun; Swami, Ananthram; Porta, Thomas F. La

doi:10.1109/tpds.2014.2311814

Cited by 8 publications

(8 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…7, we plot FP CRGT and FN CRGT , with OPT2 th ¼15 for random network, and with OPT2 th ¼7 for network with Fig.7.False positives and false negatives due to this correlation netCSI overestimates the number of failed objects. Using this insight, we propose an adaptive algorithm that works for both independent and large-scale failures in our subsequent work [25].…”

Section: Effect Of Varying Number Of Reporting Nodesmentioning

confidence: 99%

An Analytical Survey on Diagnosis Algorithm in Generic Defect Large-Scale Failures in Computer Networks

Vishwakarma

Ansari

2018

IJARCSSE

View full text Add to dashboard Cite

Abstract-We present a framework and a set of algorithms for determining faults in networks when large scale outages occur. The design principles of our algorithm, netCSI, are motivated by the fact that failures are geographically clustered in such cases. We address the challenge of determining faults with incomplete symptom information due to a limited number of reporting nodes. netCSI consists of two parts: a hypotheses generation algorithm, and a ranking algorithm. When constructing the hypothesis list of potential causes, we make novel use of positive and negative symptoms to improve the precision of the results. In addition, we propose pruning and thresholding along with a dynamic threshold value selector, to reduce the complexity of our algorithm. The ranking algorithm is based on conditional failure probability models that account for the geographic correlation of the network objects in clustered failures. We evaluate the performance of netCSI for networks with both random and realistic topologies. We compare the performance of netCSI with an existing fault diagnosis algorithm, MAX-COVERAGE, and demonstrate an average gain of 128 percent in accuracy for realistic topologies. [7]. Massive outages tend to create faults at multiple components that are geographically close to each other. We call these failures clustered failures. Until now, the prior work in the area of fault diagnosis has focused on independent failures [1], [3], [7]. The performance of these algorithms degrades when applied to clustered failures. In this paper, we propose netCSI, a new algorithm that is designed to effectively identify faulty network components under clustered failures. To show the benefits of our algorithm, we compare it with an existing algorithm that is proposed for independent failures. netCSI determines possible causes of large-scale failures using a knowledge base and end-to-end symptom information. The knowledge base contains information about possible paths between different source-destination pairs and the inferred topology of the network. The end-to-end symptoms reflect end-to-end connectivity or disconnectivity in the network and are observed when a failure occurs. These symptoms include both negative information, such as which source-destination pairs are disconnected, as well as positive information, such as which source-destination pairs can still communicate. Keywords-Fault diagnosis, large-scale network failures, incomplete information, clustered failures. I. INTRODUCTIONOnce a clustered failure occurs, netCSI uses the knowledge base and symptoms to generate a list of possible causes of the outage, called the hypothesis list. Then, a ranking algorithm is applied to the hypothesis list to rate the possible causes.The main assumption in the existing fault diagnosis algorithms [1], [2], [7] is that complete and accurate information is available at the network manager. However, during large-scale failures, it is very unlikely that complete end-toend symptom information will be available, because reporting nodes ...

show abstract

Section: Effect Of Varying Number Of Reporting Nodesmentioning

confidence: 99%

An Analytical Survey on Diagnosis Algorithm in Generic Defect Large-Scale Failures in Computer Networks

Vishwakarma

Ansari

2018

IJARCSSE

View full text Add to dashboard Cite

show abstract

“…Tati et al [27] develop algorithms to diagnose large-scale, clustered failures from incomplete symptoms and to more accurately diagnose both independent and clustered failures. Lim et al [21] mine logs of large enterprise telephony systems.…”

Section: Related Workmentioning

confidence: 99%

Digging deeper into cluster system logs for failure prediction and root cause diagnosis

Ren

McKee

et al. 2014

2014 IEEE International Conference on Cluster Computing (CLUSTER)

View full text Add to dashboard Cite

As the sizes of supercomputers and data centers grow towards exascale, failures become normal. System logs play a critical role in the increasingly complex tasks of automatic failure prediction and diagnosis. Many methods for failure prediction are based on analyzing event logs for large scale systems, but there is still neither a widely used one to predict failures based on both non-fatal and fatal events, nor a precise one that uses fine-grained information (such as failure type, node location, related application, and time of occurrence). A deeper and more precise log analysis technique is needed.We propose a three-step approach to draw out event dependencies and to identify failure-event generating processes. First, we cluster frequent event sequences into event groups based on common events. Then we infer causal dependencies between events in each event group. Finally, we extract failure rules based on the observation that events of the same event types, on the same nodes or from the same applications have similar operational behaviors. We use this rich information to improve failure prediction. Our approach semi-automates diagnosing the root causes of failure events, making it a valuable tool for system administrators.Index Terms-event causal dependency inference; failure prediction; root cause diagnosis; large-scale cluster systems;

show abstract

“…Several basic ideas are based on time window [13], frequent pattern [8] and call graph [9]. Some work combined these ideas together and point out that the system may change so often and provide a probability-based results [10].…”

Section: Related Workmentioning

confidence: 99%

A Black-Box Approach for Detecting the Failure Traces

Meng

Luan

et al. 2014

Trustworthy Computing and Services

View full text Add to dashboard Cite

Abstract. Detecting failure traces can help system administrators timely recover from those failures and avoid them afterwards. For system managers, it is not difficult to detect whether a failure is currently occurring, because they only concern about several key measurements. If these measurements exceed the normal threshold, a failure event should be generated. But it is much more complicated to detect the failure traces which represented as failure related events. Because these failure traces may last for quite a long time and effect many components. Furthermore, current distributed system adds and removes new components so quickly that administrators may not have enough time and knowledge to set monitoring threshold for each of them. Based on these problems, we propose our FTD system. We first compare each component's historical state and get outlier states as anomalous event. And then, combined with the failure event that the system provided, we detect the event correlations between failure events and anomalous events as failure traces. A network intrusion benchmark KDD99 is used to evaluate our work and we achieve good performances.

show abstract

Adaptive Algorithms for Diagnosing Large-Scale Failures in Computer Networks

Cited by 8 publications

References 13 publications

An Analytical Survey on Diagnosis Algorithm in Generic Defect Large-Scale Failures in Computer Networks

An Analytical Survey on Diagnosis Algorithm in Generic Defect Large-Scale Failures in Computer Networks

Digging deeper into cluster system logs for failure prediction and root cause diagnosis

A Black-Box Approach for Detecting the Failure Traces

Contact Info

Product

Resources

About