Monocular-Vision-Based Precise Runway Detection Applied to State Estimation for Carrier-Based UAV Landing

Abstract

Improving the level of autonomy during the landing phase helps promote the full-envelope autonomous flight capability of unmanned aerial vehicles (UAVs). Aiming at the identification of potential landing sites, an end-to-end state estimation method for the autonomous landing of carrier-based UAVs based on monocular vision is proposed in this paper, which allows them to discover landing sites in flight by using equipped optical sensors and avoid a crash or damage during normal and emergency landings. This scheme aims to solve two problems: the requirement of accuracy for runway detection and the requirement of precision for UAV state estimation. First, we design a robust runway detection framework on the basis of YOLOv5 (you only look once, ver. 5) with four modules: a data augmentation layer, a feature extraction layer, a feature aggregation layer and a target prediction layer. Then, the corner prediction method based on geometric features is introduced into the prediction model of the detection framework, which enables the landing field prediction to more precisely fit the runway appearance. In simulation experiments, we developed datasets applied to carrier-based UAV landing simulations based on monocular vision. In addition, our method was implemented with help of the PyTorch deep learning tool, which supports the dynamic and efficient construction of a detection network. Results showed that the proposed method achieved a higher precision and better performance on state estimation during carrier-based UAV landings.

VALNet: Vision-Based Autonomous Landing with Airport Runway Instance Segmentation

“…As mentioned in the references, a precise runway detection method based on YOLOv5 is proposed in [26], a benchmark for airport detection using Sentinel-1 SAR (Synthetic Aperture Radar) is introduced in [66], and several detection approaches based on deep learning principles are presented in [14,[67][68][69]. Presently, only private aviation companies are endeavoring to research deep learning solutions of runway detection by forward-facing cameras located on the aircraft's nose or wings, which has achieved notable success in vision-based autonomous landing systems such as the capabilities of the autonomous taxiing, taking off, and landing of Airbus [70] and the Daedalean project [71].…”

“…Typical methods include convolutional neural networks (CNNs), fully convolutional networks (FCNs) [23], and advanced architectures like U-Net [24] and Mask R-CNN [25]. These methods leverage hierarchical feature extraction and end-to-end learning to effectively capture the complex patterns and features of runway images [26]. However, they also face three main challenges that need to be addressed to achieve reliable runway segmentation for autonomous landing:…”

An Innovative Runway Landing Path Detection using UAV Implementation of the K-Means Clustering Algorithm

“…The proliferation of Unmanned Aerial Systems (UAS) has led to the emergence of numerous unique programs tailored to small businesses. These programs capitalize on advancements in UAV technology, such as fixed-wing (4) and multirotor platforms, for tasks like airborne https://www.indjst.org/ tracking, image processing, and object detection (5) . While object detection models like Faster R-CNN, YOLO (6) and morphological fusion (7) excel at identifying objects in images, they often require extensive labeled training data and may be sensitive to annotation quality, potentially impacting their performance.…”

An Enhanced Aircraft Carrier Runway Detection Method Based on Image Dehazing

“…Therefore, as a crucial aspect of the entire mission execution of shipborne CUAVs, CUAV landing has gradually become a popular research field in recent years. Typically, CUAV landing methods mostly rely on sensors such as vision, altitude, and laser, with vision sensors gradually becoming the most commonly used sensor type in this neighborhood due to their low cost and high accuracy [1]. One of the most crucial subtasks in landing missions is runway line detection and positioning.…”