Fault Localization in Large-Scale Network Policy Deployment

Abstract

The recent advances in network management automation and Software-Defined Networking (SDN) are easing network policy management tasks. At the same time, these new technologies create a new mode of failure in the management cycle itself. Network policies are presented in an abstract model at a centralized controller and deployed as low-level rules across network devices. Thus, any software and hardware element in that cycle can be a potential cause of underlying network problems. In this paper, we present and solve a network policy fault localization problem that arises in operating policy management frameworks for a production network. We formulate our problem via risk modeling and propose a greedy algorithm that quickly localizes faulty policy objects in the network policy. We then design and develop SCOUT-a fully-automated system that produces faulty policy objects and further pinpoints physical-level failures which made the objects faulty. Evaluation results using a real testbed and extensive simulations demonstrate that SCOUT detects faulty objects with small false positives and false negatives.

Towards Internet-Scale Convolutional Root-Cause Analysis with DIAGNET

“…in terms of peering-points), a property that we call network and service agnosticism. This largely departs from common root cause analysis relying on network tomography [11]- [13], bespoke methods for data centers [14], [15] and Software-Defined-Networking [16], [17].…”

“…Numerous works have been performed in enterprise networks and datacenters, where the full network topology is known (e.g. Clos-like topologies or SDN-driven networks [16], [17], [36]). This topology information allows network tomography techniques to be applied [11]- [15], [37], [38], pinpointing faulty links or components accurately at scale despite complex dependencies between components [39].…”

Tardis

“…Tardis stands to benefit from a decrease in bugs: In the event of a (less-likely) fault, Tardis will have more fault-free code paths to use in recovering from the fault. Prior efforts also focused on localizing problems in SDNs [28,72,73,80,86], but these are offline techniques and do not attempt to recover from or circumvent a fault. Automated Program Repair.…”

DiagNet: towards a generic, Internet-scale root cause analysis solution

“…We argue therefore that an Internet-scale root cause analysis method should not make any assumptions on the hidden network topology, a property that we call network topology agnosticism. This largely departs from common root cause analysis that relies on network tomography [16]- [19] or bespoke methods for data centers [20], [21] and Software-Defined-Networking [22], [23].…”

“…We note that this method can also be used in conjunction with passive monitoring for bootstrapping a system [24], or testing hypotheses [40]. Numerous works have been performed in enterprise networks and datacenters, where the full network topology is known (e.g Clos-like topologies in Azure, or SDN-driven networks [22], [23]). This topology information allows network tomography techniques to be applied [16]- [21], [41], [42], pinpointing faulty links or components accurately at scale despite complex dependencies between components [43].…”