A real-time photogrammetric algorithm for sensor and synthetic image fusion with application to aviation combined vision

Lebedev, Maksim; Stepaniants, D. G.; Komarov, D. V.; Выголов, О. В.; Vizilter, Yu. V.; Zheltov, Sergey Yu.

doi:10.5194/isprsarchives-xl-3-171-2014

Cited by 6 publications

(1 citation statement)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…11 Horn in German Aerospace Center (DLR) used fuzzy logic-based method. 12 The researchers in Russia State Research Institute of Aviation Systems (GosNIIAS) proposed a morphological registration method, 13,14 while the researchers in India National Aerospace Laboratory (CSIR-NAL) used differential GPS (DGPS) instruments to provide accurate navigation and positioning for ESVS system. 15 However, these traditional methods rely heavily on high-precision INS and ground-based augmentation system (GBAS), they can neither be used to low-cost general aviation nor robustly run in GPS-denied condition.…”

Section: Introductionmentioning

confidence: 99%

Visual–inertial fusion-based registration between real and synthetic images in airborne combined vision system

Zhang

Zhai

Niu

et al. 2019

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

Combined vision system is a perspective display concept to enhance a situation awareness of the pilots during aircraft landing, which integrates a real 2-D image captured from forward-looking infrared camera with a synthetic 3-D image derived by the aircraft pose and terrain database. However, the inertial measured errors significantly affect the conformal display of combined vision. This article proposes a novel method for real and synthetic images registration based on visual-inertial fusion. It includes the following key steps: (1) detect and extract the real runway features from forwardlooking infrared image; (2) generate the synthetic runway features simultaneously; (3) set up vision measurement model with real and synthetic runway features; (4) integrate inertial data and visual observations in the square-root unscented Kalman filter; (5) create a synthetic 3-D scene by the filtered pose data and integrate it with a real 2-D image. The experimental results demonstrate that our method can guarantee the conformal display of combined vision system in GPS-denied and low visibility conditions.

show abstract

Section: Introductionmentioning

confidence: 99%

Visual–inertial fusion-based registration between real and synthetic images in airborne combined vision system

Zhang

Zhai

Niu

et al. 2019

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

show abstract

Infrared Image Enhancement by Multi-Modal Sensor Fusion in Enhanced Synthetic Vision System

Cheng

Han

et al. 2020

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

ESVS (Enhanced Synthetic Vision System) employs multi-modal sensor fusion to provide equivalent out-of-cabin view for pilots. By using such device, pilots can identify the runway and obstacles even in low visibility weather condition during approach and landing. Previously, short-wave infrared video was used as real-time image sensor in our developed ESVS. While, compared to visible image, infrared image is not easy to understand. Moreover, the image quality of infrared degrades to a considerable extent in haze weather condition. To improve the infrared image visibility in ESVS, we proposed a multi-modal sensor fusion strategy by combining real-time infrared video and previous recorded visible video in the clear weather condition. During image fusion, considerable color detail of the visible image is injected to infrared image to ameliorate the infrared image quality. Furthermore, the proposed fusion method has two extra advantages. First, the important tagged information including runway and obstacles in visible image are transferred to infrared video, and second, the missing information in infrared vision can be complemented. To evaluate our proposed method, we use the Y-12F aircraft equipped with both visible and infrared video camera for flight test data collection. By collecting visible and infrared video in clear and haze weather conditions, as well as necessary navigation data, we carry out the final image fusion processing. Experimental results show that the fused video frame has enhanced image quality and improved readability for the pilots, which will significantly promote the pilots’ situation awareness during approach and landing. Thus, the proposed idea for fusion of multi-sensor image has the great potential to be applied to other cockpit display systems.

show abstract

Semantic image segmentation for information presentation in enhanced vision

et al. 2017

View full text Add to dashboard Cite

A real-time photogrammetric algorithm for sensor and synthetic image fusion with application to aviation combined vision

Cited by 6 publications

References 2 publications

Visual–inertial fusion-based registration between real and synthetic images in airborne combined vision system

Visual–inertial fusion-based registration between real and synthetic images in airborne combined vision system

Infrared Image Enhancement by Multi-Modal Sensor Fusion in Enhanced Synthetic Vision System

Semantic image segmentation for information presentation in enhanced vision

Contact Info

Product

Resources

About