Fusion of Enhanced and Synthetic Vision System Images for Runway and Horizon Detection

Abstract

UAV network operation enables gathering and fusion from disparate information 10 sources for flight control in both manned and unmanned platforms. In this investigation, a novel 11 procedure for detecting runways and horizons as well as enhancing surrounding terrain is 12 introduced based on fusion of enhanced vision system (EVS) and synthetic vision system (SVS) 13 images. EVS and SVS image fusion has yet to be implemented real-world situations due to signal 14 misalignment. We address this through a registration step to align the EVS and SVS images. 15Four fusion rules combining discrete wavelet transform (DWT) sub-bands are formulated, 16implemented and evaluated. The resulting procedure is tested on real EVS-SVS image pairs and 17 pairs containing simulated turbulence. Evaluations reveal that runways and horizons can be 18 detected accurately even in poor visibility. Furthermore, it is demonstrated that different aspects 19 of the EVS and SVS images can be emphasized by using different DWT fusion rules. The 20 procedure is autonomous throughout landing, irrespective of weather. We believe the fusion 21 architecture developed holds promise for incorporation into head-up displays (HUDs) and UAV 22 remote displays to assist pilots landing aircraft in poor lighting and varying weather. The 23 algorithm also provided a basis rule selection in other signal fusion applications. 24 26 27 29 accidents are reported to occur during the final approach and landing stages [1]. While 30 instrumented landing systems have successfully been implemented to provide precise landing 31 guidance, they are not available at all airports. Furthermore, smaller aircraft and fixed wing 32 unmanned air vehicles (UAVs) often land in remote locations with only small runway strips 33 available. Thus, there is a clear need to assist pilots and remote operators using visual flight 34 landing aids to detect runways accurately in varying weather conditions. Readily available 35 imaging systems offer obvious potential to address this issue, but a single mode of image capture 36 often does not fully convey all vital landing information in time critical situations. Fusion of ground 37 sensor arrays (e.g. infrared cameras [2]) have been proposed to provide real-time input to UAVs, in 38 particular in the lack of GPS information [3]. Such systems can be enhanced with through the 39 fusion of image information from disparate heterogeneous sensors in real-time. 40The convolution of output acquired from multiple sensors capturing complementary 41 information has received significant attention in recent years. Techniques for fusing such 42 information from images have been applied to a diverse range of fields, including: medical imaging 43 [4, 5], remote sensing [6,7], intelligent transport [8,9], surveillance [10], low altitude remote sensing 44 [11], and color visibility enhancement [12]. Very recent approaches [13] have introduced weighted 45 fusion strategies aimed at improving the robustness of detection robustness against the size of 46 object. Su...

Vision-Assisted Landing Method for Unmanned Powered Parachute Vehicle Based on Lightweight Neural Network

“…Author Name: Preparation of Papers for IEEE Access (February 2017) 8 VOLUME XX, 2017 the initial offset of the UPPV test platform is large owing to the error in GPS navigation when it enters the visual landing phase. However, the offset can be corrected to 0 within 15s with the effect of the autonomous landing visual positioning system, and the comprehensive recognition accuracy of the landing runway was higher than 97.47% for each test during the entire visual landing process.…”

“…Nazir et al [7] acquired landing runway images using an airborne camera, and estimated the exact location of the landing runway using edge detection algorithms. Fadhil et al [8] proposed a sensor fusion algorithm based on the Hough transform to detect runways. Zhang et al [9] designed a fuzzy Canny edge extraction algorithm and combined it with a curve-fitting method to obtain the centerline of the runway.…”

Visual Landing Based on the Human Depth Perception in Limited Visibility and Failure of Avionic Systems

“…However, landing is still one of the most accident-prone ight phases, with a relatively high percentage of fatal and nonfatal air accidents [1,2]. It has been reported that almost half of plane crashes occur in the approach and nal landing stages [3]. If visibility is limited, the pilot will land using avionics and precision instrument landing systems.…”

“…However, it did not implement the accommodation cue in visual cues. Other studies have combined synthetic vision systems (SVS) and enhanced flight vision systems (EFVS) [3]. Artificial vision data are utilized to identify dynamic objects on the runway surface [22].…”

Optical Navigation Sensor for Runway Relative Positioning of Aircraft during Final Approach

“…In [ 35 ], the authors present a family of edge detectors for various orientations. Beyond line feature detection, the registration of a synthetic image with known 6D pose [ 36 , 37 ] and registration of real images from database [ 38 ] are also applied to solve the runway detection problem.…”