NAS-Wide Performance: Impact of Select Uncertainty Factors and Implications for Experimental Design

Abstract

Performance estimates produced by a National Airspace System (NAS)-wide simulation models vary due to the complexity and amount of variability that occurs within the NAS. One area of modeling variability which current NAS-wide simulation models attempt to compensate for is the variation in delay occurring across days. This is typically accomplished through the use of a carefully-selected set of days seeking to be "representative" of the NAS performance across a given year. These days are referred to as design days. Current practices model each design day once, with averaging across all design days to yield annual estimates of performance. The concern with this process is that each design day represents one specific instance of what could have happened in the NAS and does not consider the many small daily variations that could have a potentially significant impact. Also, creating design days is an interactive and time-consuming process, so simply creating additional design days to improve the confidence of the model results is not always economically feasible. This paper determines the impact that intra-day perturbations due to four factors within a NAS-wide model (air carrier delay, runway configuration changes, sector workload limits and program rate forecasts associated with ground delay programs) have on NASwide simulation results. This paper also determines the combinations of design days and iterations per design day required to achieve convergence of NAS-wide estimates for a given confidence level when the four factors within the model are perturbed. The conclusions of this paper are that averaging across design days provides a high level of confidence in the results up to a point but for even higher levels of confidence it becomes important to include iterations in the experimental design. The four factors added an additional 37% NAS-wide delay to the model results. We expected the four factors to increase delay in model results as some of these factors were not previously modeled and are new forms of delay. The factor that contributed the most to the variability of NAS-wide delay was the program rate forecasts associated with ground delay programs.