On the Use of Emulab Testbeds for Scientifically Rigorous Experiments

Abstract

Abstract-Internet is considered a Critical Infrastructure (CI) that is vital for both the economy and the society. Disruptions caused by natural disasters, malicious human actions and even hardware failure pose serious risks. Emulation testbeds are increasingly used to study the Internet in order to improve protection and response mechanisms. These are frequently considered more adequate than software simulators to realistically recreate the complex behavior of networks. In this paper we study how testbeds based on the Emulab software can be used to conduct scientifically rigorous experiments, specifically in terms of: a) experiment fidelity, b) repeatability, c) measurement accuracy, and d) interference.Our study, which is based on extensive experimentation on different testbeds, indicates that the current trend of using emulation testbeds is justified as both realistic and efficient. We show that Emulab-based experiments are representative of real systems in terms of emerging behavior (qualitative) and that repeatable experiments are possible. The main contribution of this tutorial article is that based on experimental results we identified caveats and provided insights to significant configuration parameters and limitations that are further elaborated into a set of guidelines that any Emulab user should be aware of. Then, he/she can decide about the importance of each guideline in the context of a specific study and experiment.

Cyber Security Impact on Power Grid Including Nuclear Plant

“…• an experimental platform for resilience, security and stability research, called Experimental Platform for Internet Contingencies (EPIC) [11], which supports the security assessment of cyber-physical systems. The EPIC test-bed can efficiently recreate realistic network topologies and conditions (e.g.…”

“…a power grid, and can interact with the emulated network test-bed. The architecture of EPIC suggests the use of an emulation testbed based on the Emulab software [11] in order to recreate the cyber part of NCIs, e.g., servers and corporate network, and the use of software simulation (AMICI) for the physical components, e.g., power grid and nuclear plant.…”

A cyber-physical experimentation environment for the security analysis of networked industrial control systems

“…This provides an efficient, safe and low-cost approach with fast and accurate analysis capabilities. For the cyber part of NICS this work adopted an emulation approach based on Emulab [5,6]. This approach is well-established in the field of cyber security [20] and was chosen in order to overcome the major difficulties that rise while trying to simulate how ICT components behave under attacks or failures.…”

“…In other words the Emulab software configures the physical topology in a way that it emulates the virtual topology as accurately as possible [6]. Besides the process network, the cyber layer also includes the control logic code, that in the real world is implemented by PLCs.…”

“…The proposed framework uses simulation for the physical components and an emulation testbed based on Emulab [5,6] in order to recreate the cyber part of NICS, e.g. SCADA servers, corporate network.…”