A NEAT way to test-driven network management

Abstract

The increasing softwarization of network infrastructures introduces an important challenge for network configuration. On the one hand, the growth of the network configuration space as a result of new device types and the expanding inter-dependence of network service components, increases the network configuration complexity. On the other hand, new service deployment architectures lack mechanisms to validate the impact of service configuration on network resilience. Network operators need to adopt new mechanisms to validate and verify network configuration changes, inspired by popular Continuous Integration/Continuous Development (CI/CD) mechanisms. This paper introduces Network Emulation-based Automated Testing (NEAT), an automated testing framework for network configuration. NEAT allows network managers to define network topologies and tests through YAML files and run realistic network topologies and tests. Furthermore, network managers can control the fidelity of their network tests and bound the execution time of testing suites, as well as exploit parallelization of modern servers to speedup test execution.

Improving Intent Correctness with Automated Testing

“…However, as the scope of an IBN system expands, so does the support for failure scenarios which reduces the effectiveness of modeling. Optimization and validation mechanisms should be coupled with network emulation [17], [11] and verification mechanisms [18] to expand testing coverage and incorporate complex elements that cannot be captured by the mathematical modeling that typically underpins these mechanisms. To reflect these requirements, we propose an extension in the IETF lifecycle model and include a "test/verify" stage in the lifecycle prior to the provision stage.…”

“…NEAT [11] is an open-source network testing framework for network configuration. Network managers and developers can use NEAT to re-create realistic network topologies with custom network configurations, run asynchronous network tests and collect detailed information about test outcomes.…”

“…In this paper, we argue that effective adoption of IBN [11], which uses realistic device models and network emulation to validate device configuration correctness. The proposed testing platform utilizes connectivity intent requirements to automatically generate testing suites tailored to the network configuration, and can deliver integration testing between legacy and programmable devices.…”

NES: Towards Lifecycle Automation for Emulation-Based Experimentation

“…For example, Network Developer Operations (Net-DevOps) and Continuous Integration/Continuous Development (CI/CD) operationalize network testing, deployment, and monitoring, in an effort to achieve automation levels, typically found in the cloud-software domain. The success of Net-DevOps depends on the ability to support multi-dimensional testing, and network emulation can realize suitable testing environments, facilitating integration tests for network configurations in a CI/CD pipeline [8]. Similarly, network twins propose the fusion of network models (simulation and emulation) with live network data as a way to predict the behavior and support the management decision processes.…”

“…Recent research efforts explore the integration of network emulation in automated integration pipelines. Network Emulation-based Automated Testing (NEAT) [8] testing platform that uses the Mininet API and leverages containerization to perform automated testing in cloud CI/CD environments. However, the emergence of NEaaS [27] looks to be the most direct way for emulation to adopt the technologies required for the demands of modern workflows.…”