Optimal solutions to Markov decision problems may be very sensitive with respect to the state transition probabilities. In many practical problems, the estimation of these probabilities is far from accurate. Hence, estimation errors are limiting factors in applying Markov decision processes to real-world problems.We consider a robust control problem for a finite-state, finite-action Markov decision process, where uncertainty on the transition matrices is described in terms of possibly nonconvex sets. We show that perfect duality holds for this problem, and that as a consequence, it can be solved with a variant of the classical dynamic programming algorithm, the "robust dynamic programming" algorithm. We show that a particular choice of the uncertainty sets, involving likelihood regions or entropy bounds, leads to both a statistically accurate representation of uncertainty, and a complexity of the robust recursion that is almost the same as that of the classical recursion. Hence, robustness can be added at practically no extra computing cost. We derive similar results for other uncertainty sets, including one with a finite number of possible values for the transition matrices.We describe in a practical path planning example the benefits of using a robust strategy instead of the classical optimal strategy; even if the uncertainty level is only crudely guessed, the robust strategy yields a much better worst-case expected travel time. Notation P > 0 or P 0 refers to the strict or nonstrict componentwise inequality for matrices or vectors. For a vector p > 0, log p refers to the componentwise operation. The notation 1 refers to the vector of ones, with size determined from context. The probability simplex in R n is denoted n = p ∈ R n + p T 1 = 1 , while n is the set of n × n transition matrices (componentwise nonnegative matrices with rows summing to one). We use to denote the support function of a set ⊆ R n , with for v ∈ R n , v = sup p T v p ∈ .

We describe a general con ict detection/resolution scheme, focusing on the con ict detection component for a pair of aircraft ying at the same altitude. The proposed approach is formulated in a probabilistic framework, thus allowing uncertainty in the aircraft positions to be explicitly taken into account when detecting a potential con ict. The computational issues involved in the application of the proposed con ict alerting system are addressed by resorting to randomized algorithms. Finally, the validation of the proposed detection scheme is performed by Monte Carlo simulation on a stochastic ODE model of the aircraft motion. Our simulations show very promising results. The detection scheme will be used as the basis for a con ict resolution scheme in forthcoming work.

Much of the delay in the US National Airspace System (NAS) arises from convective weather. One major objective of our research is to take a less conservative route, where we take a risk of higher delay to atkin a better expected delay, instead of avoiding the bad weather zone completely. We address the single aircraft problem using a Markov decision process model and a stochastic dynamic programming algorithm, where the evolution of the weather is modeled as a stationary Markov chain. Our solution provides a -dynamic routing strategy for an aircraft that minimizes the expected delay. A significant improvement in delay is obtained by using our methods over the traditional methods. In addition, we propose an algorithm for dynamic routing where the solution is robust with respect to the estimation errors of the storm probabilities. To the Bellman equations, which are derived in solving the dynamic routing strategy of an aircraft, we add a further requirements: we assume that the transition probabilities are unknown, but bounded within a convex set. The uncertainty described in our approach is based on likelihood hctions. This makes the robust Dynamic Programming (DP) "tractable" (that is, not much more complicated than the original DP) and yet not conservative. Our algorithm optimizes the performance of the system, given there are errors in the estimation of the probabilities of the storms. OverviewAir traffic delay due to convective weather has grown rapidly over the last few years. Accorrlmg to the FAA, flight delays have increased by more than 58 percent since 1995, cancellations by 68 percent. The delay distribution can be described in the figure below:

No abstract

In this paper, we study the problem of designing optimal coordinated maneuvers for multiple aircraft conflict resolution. We propose an energy function to select among all the conflict-free maneuvers the optimal one. The introduced cost function incorporates a priority mechanism that favors those maneuvers where aircraft with lower priority assume more responsibility in resolving the predicted conflicts. The energy-minimizing resolution maneuvers may involve changes of heading and speed, as well as of altitude. However, vertical maneuvers are penalized with respect to horizontal ones for the sake of passenger comfort. A geometric construction and a numerical algorithm for computing the optimal resolution maneuvers are given in the two aircraft case. As for the multiaircraft case, an approximation scheme is proposed to compute a suboptimal two-legged solution. Extensive examples are presented to illustrate the effectiveness of the proposed algorithms.