Experimental evaluation of wireless simulation assumptions

Abstract

All analytical and simulation research on ad hoc wireless networks must necessarily model radio propagation using simplifying assumptions. We provide a comprehensive review of six assumptions that are still part of many ad hoc network simulation studies, despite increasing awareness of the need to represent more realistic features, including hills, obstacles, link asymmetries, and unpredictable fading. We use an extensive set of measurements from a large outdoor routing experiment to demonstrate the weakness of these assumptions, and show how these assumptions cause simulation results to differ significantly from experimental results. We close with a series of recommendations for researchers, whether they develop protocols, analytic models, or simulators for ad hoc wireless networks.Mobile ad hoc networking (MANET) has become a lively field within the past few years. Since it is difficult to conduct experiments with real mobile computers and wireless networks, nearly all published MANET articles are buttressed with simulation results. Many such simulations may be based on overly simplistic assumptions, however; a recent article in IEEE Communications warns that "An opinion is spreading that one cannot rely on the majority of the published results on performance evaluation studies of telecommunication networks based on stochastic simulation, since they lack credibility" [10]. It then proceeded to survey 2200 published network simulation results to point out systemic flaws.Although not every simulation study needs to use the most detailed radio model available, nor explore every variation in the wide parameter space afforded by a complex model, there are real risks to protocol designs based on overly simple models of radio propagation. As the nodes move in an ad hoc network, the connectivity graph changes over time, but these changes depend significantly on variations in the range due to antenna differences, elevation (extending range), and obstacles (restricting range).We recognize that the MANET research community is increasingly aware of the limitations of its simplifying assumptions. Our goal in this paper is to make a constructive contribution to the MANET community by a) clearly identifying these assumptions and quantitatively demonstrating their weaknesses, b) comparing simulation results to experimental results to identify how simplistic radio models can lead to misleading results in ad hoc network research, c) contributing a real dataset that should be easy to incorporate into simulations, and d) listing recommendations for the designers of protocols, models, and simulators.Due to limited space, this paper presents only the main points. We present all of the details in an extended Technical Report [6]. MODELS USED IN RESEARCHThe simplest radio models are based on distance across flat terrain; radio communications are received perfectly within some circular "range" and not at all outside of that range. Real radios, including those used in the popular Berkeley Motes, demonstrate a strikingly non-uniform non-...

A Model to Improve the Accuracy of WSN Simulations

“…These assumptions lead to simulation results that differ significantly from experimental results [3].…”

Workload Modeling for Computer Systems Performance Evaluation

“…But note that a complete user mobility model may depend not only on user behavior, but also on physical characteristics of the system. Specifically, Kotz et al [414] show how the movement among different access points may depend on radio propagation characteristics, because the nearest access point may not always provide the strongest signal due to blocking or fading effects. In particular, they point out that common assumptions used in models and simulations, such as that all access points provide full coverage in perfectly circular areas with the same radius, are way too simplistic.…”

Co-simulation of wireless networked control systems over mobile ad hoc network using SIMULINK and OPNET

“…Figure 3 shows the comparison of computer simulation models and real world experiments [3], [33], [34]. The radio channel used in this paper implements model 1 expressed in (7) where P r is the received wireless power, d…”

“…To reflect the open field environment, path loss exponent, β=2.8 and fading effect, X dB =6dB have been implemented. Thirteen MANET nodes produce the same node density as presented in [3], [33], [34]. To observe the effect of the number of network nodes, five network sizes, i.e., six, ten, thirteen, twenty and twenty six nodes have been considered in section III.…”