No abstract

In this paper, we explore the role of simulators and testbeds in the development procedure of protocols or applications for Wireless Sensor Networks (WSNs) and Internet of Things (IoT). We investigate the complementarity between simulation and experimentation studies by evaluating latest features available among open testbeds (e.g., energy monitoring, mobility). We show that monitoring tools and control channels of testbeds allow for identification of crucial issues (e.g., energy consumption, link quality) and we identify some opportunities to leverage those real-life obstacles. In this context, we insist on how simulations and experimentations can be efficiently and successfully coupled with each other in order to obtain reproducible scientific results, rather than sole proofs-of-concept. Indeed, we especially highlight the main characteristics of such evaluation tools that allow to run multiple instances of a same experimental setup over stable and finely controlled components of hardware and real-world environment. For our experiments, we used and evaluated the FIT IoT-LAB facility. Our results show that such open platforms, can guarantee a certain stability of hardware and environment components over time, thus, turning the unexpected failures and changing parameters into core experimental parameters and valuable inputs for enhanced performance evaluation.

Validation of protocols and mechanisms is an essential step to the development of object networks in critical domains. Most papers still provide evaluation either obtained through theoretical analysis or simulations campaigns. Yet, simulators and formal models fail to precisely reproduce the unique specificities of the deployment environments those networks have to evolve in. Also, by putting no limits to code complexity and execution, those tools prevent users to apprehend the actual limits of WSN nodes and to propose realistic communication protocols and applications. In this paper, we highlight to what extent the addition of experimentations can significantly improve the value of performance evaluation campaigns. Along with the recent tendency to have algorithmic and protocol proposals facing real environments, it is questionable whether the so obtained results should be considered as scientific or empirical ones. In the former case, reproducibility, stability over time, topology management to cite a few, are a must have for testbeds and real deployments that are used. In the latter, the results should be viewed as a proof-of-concept only, far from independent of the used hardware and encountered conditions at the experimentation time but still critical from the development cycle standpoint. Through some experiments over the IoT-LAB testbed, we aim at demonstrating to what extent some of the simulation setup and conditions from reality could be emulated. We also provide insight on how to obtain the best out of it in a quick and efficient manner. We show that such testbeds would satisfy many expectations (e.g. scientific tool and proof-of-concept validator), thus minding and bridging some of the gaps between theory and practice in WSN. To this end, we here give an overview of available simulation tools, and guidelines on how to transpose simulation setups to the open large-scale IoT-LAB platform.

International audiencePerformance analysis of newly designed solutions is essential for efficient Internet of Things and Wireless Sensor Network (WSN) deployments. Simulation and experimental evaluation practices are vital steps for the development process of protocols and applications for wireless technologies. Nowadays, the new solutions can be tested at a very large scale over both simulators and testbeds. In this paper, we first discuss the importance of repeatable experimental setups for reproducible performance evaluation results. To this aim, we present FIT IoT-LAB, a very large-scale and experimental testbed, i.e., consists of 2769 low-power wireless devices and 127 mobile robots. We then demonstrate through a number of experiments conducted on FIT IoT-LAB testbed, how to conduct meaningful experiments under real-world conditions. Finally, we discuss to what extent results obtained from experiments could be considered as scientific, i.e., reproducible by the community