Hardware acceleration of lucky-region fusion (LRF) algorithm for imaging

Abstract

Lucky-region" fusion (LRF) is a synthetic imaging technique that has proven successful in enhancing the quality of images distorted by atmospheric turbulence. The LRF algorithm extracts sharp regions of an image obtained from a series of short exposure frames, and fuses the sharp regions into a final, improved image. In our previous research, the LRF algorithm had been implemented on a PC using the C programming language. However, the PC did not have sufficient processing power to handle real-time extraction, processing and reduction required when the LRF algorithm was applied to real-time video from fast, high-resolution image sensors rather than single picture images. This document describes a hardware implementation of the LRF algorithm on a VIRTEX-7 field programmable gate array (FPGA) to achieve real-time image processing. The novelty in our approach is the creation of a "black box" LRF video processing system with a general camera link input, a user controller interface, and a camera link or DVI video output. We also describe a custom hardware simulation environment we have built to test our LRF implementation.

Hardware acceleration of lucky-region fusion (LRF) algorithm for high-performance real-time video processing

“…4,7,8 The atmosphere can also cause differences in the scene from frame to frame, hindering the tracking of moving targets. 1,4 Compensating for distortions in image quality caused by turbulence is by no means a new problem, and many image processing methods have been developed to try and combat these effects. Many of these techniques were originally developed for astronomical applications, where it generally assumed that the distortions are independent of the position in the image plane, called the isoplanatic condition.…”

“…Light waves travelling through such chaotic regions of changing refractive index will undergo a complex combination of refraction and scattering, resulting in extensive spatially and temporally varying distortions in the images. 4,5 Turbulence can be comprised of geometric distortion (motion), spatial and temporal blurs, locally varying blurs, and out-of-focus blurs. 4,7,8 The atmosphere can also cause differences in the scene from frame to frame, hindering the tracking of moving targets.…”

“…4,5 Turbulence can be comprised of geometric distortion (motion), spatial and temporal blurs, locally varying blurs, and out-of-focus blurs. 4,7,8 The atmosphere can also cause differences in the scene from frame to frame, hindering the tracking of moving targets. 1,4 Compensating for distortions in image quality caused by turbulence is by no means a new problem, and many image processing methods have been developed to try and combat these effects.…”

“…Optical turbulence is the result of variations in the refractive index in the path between the camera sensor and the target. 1,2,3,4 Atmospheric characteristics such as temperature, humidity, and pressure all factor into the resulting refractive index of the air between the target and optical system. Since all these parameters are rarely modeled linearly, modelling the turbulence as a function of these parameters becomes even more challenging.…”

“…Many of these techniques were originally developed for astronomical applications, where it generally assumed that the distortions are independent of the position in the image plane, called the isoplanatic condition. 4,8,9 An early predecessor to Lucky Region Fusion (LRF) was a method called lucky frame selection, which involved selecting the frames with the highest quality from a set of randomly distorted incoming frames. 2,4,10,11 However, this approach was not viable under conditions where the isoplanatic condition did not hold due to a very low probability of finding entire frames of high quality.…”