An ETA-Based Tactical Conflict Resolution Method for Air Logistics Transportation

Abstract

Air logistics transportation has become one of the most promising markets for the civil drone industry. However, the large flow, high density, and complex environmental characteristics of urban scenes make tactical conflict resolution very challenging. Existing conflict resolution methods are limited by insufficient collision avoidance success rates when considering non-cooperative targets and fail to take the temporal constraints of the pre-defined 4D trajectory into consideration. In this paper, a novel reinforcement learning-based tactical conflict resolution method for air logistics transportation is designed by reconstructing the state space following the risk sectors concept and through the use of a novel Estimated Time of Arrival (ETA)-based temporal reward setting. Our contributions allow a drone to integrate the temporal constraints of the 4D trajectory pre-defined in the strategic phase. As a consequence, the drone can successfully avoid non-cooperative targets while greatly reducing the occurrence of secondary conflicts, as demonstrated by the numerical simulation results.

Risks of Drone Use in Light of Literature Studies

“…While [ 152 ] describes the technological or regulatory challenges for UAVs, in [ 204 , 222 ], attention is drawn to the legal-political, socio-legal approach and the implementation of regulations, highlighting the measures that should be taken before making a given technology available to the public. Some of the important areas are planning paths, avoiding obstacles, and navigation [ 143 , 154 , 156 , 188 , 189 , 190 , 191 , 192 , 193 , 194 , 195 , 196 , 197 , 198 , 199 , 200 , 201 , 202 ]. The aspects mentioned above have a significant impact on the operational safety of the drone.…”

Comprehensive Review of Drones Collision Avoidance Schemes: Challenges and Open Issues

“…Addressing this, [80] presents a two-layer resolution framework employing DRL for UAVs, enabling continuous action space and enhancing decision-making capabilities to provide adaptive avoidance strategies. In [86] RL contributions enable a drone to incorporate the time-related limitations of the 4D trajectory predetermined in the strategic phase. According to the DRL approach by [84], features that describe collisions between agents are explicitly represented as edges in a dynamic agents' social graph, and agents are encouraged to share observations in their neighborhood in an intrinsically partially observable environment.…”

“…However, it increases the computational complexity, especially as the network size grows. Reference [86] presents a technique for resolving tactical conflicts using deep reinforcement learning with non-cooperative UAVs according to a time reward set with an estimated arrival time. This approach introduces a reward criterion based on the estimated time to reach the next pre-defined waypoint.…”

“…Tactical conflict resolution techniques disregard the time limits of the predefined strategic trajectories, which may cause drones to miss their next trajectory points during tactical conflict resolution. This might exacerbate secondary conflicts and even cause a "domino effect" [86], [154]. Existing empirical studies have supported the extensive use of AI in drone However, algorithms have yet to be done without considering the primary constraints of drones, such as their high energy consumption or limited capacity for data processing.…”

Adaptive Collision Avoidance for Multiple UAVs in Urban Environments

“…In [19], the DDQN algorithm was applied to ship navigation to achieve multi-ship collision avoidance in crowded waters. Li et al designed a tactical conflict resolution method for air logistics transportation based on the D3QN algorithm, enabling UAVs to successfully avoid non-cooperative targets [20]. The value-based algorithms (e.g., D3QN and DDQN) can adapt to complex state spaces, but cannot provide satisfactory solutions for continuous control problems.…”