Fusion of interpolated frames superresolution in the presence of atmospheric optical turbulence

Hardie, Russell C.; Rucci, Michael A.; Karch, Barry K.; Dapore, Alexander J.; Droege, Douglas R.; French, Joseph C.

doi:10.1117/1.oe.58.8.083103

Cited by 12 publications

(16 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where λ is the wavelength and the path length is given by the variable L. The Fried parameter is the key parameter that governs the atmospheric optical transfer function (OTF), the PSF, and point source tilt variance. 41 It is worth noting that a small r 0 relative to the camera aperture indicates a high level of atmospheric turbulence. Warping from turbulence can be characterized by the point source angular tilt variance.…”

Section: Turbulence Characterizationmentioning

confidence: 99%

“…It is worth noting that some prior work has been done addressing the problems of turbulence and undersampling jointly using an approach based on the BM-WF method. 41 We forgo that extra complication in this study, as the emphasis here is on using machine learning for TM. The optical and turbulence parameters have been selected to closely match those in the paper 2 that describes the anisoplanatic simulator employed here.…”

Section: Turbulence Characterizationmentioning

confidence: 99%

See 1 more Smart Citation

Deep learning for anisoplanatic optical turbulence mitigation in long-range imaging

et al. 2021

Self Cite

View full text Add to dashboard Cite

We present a deep learning approach for restoring images degraded by atmospheric optical turbulence. We consider the case of terrestrial imaging over long ranges with a wide fieldof-view. This produces an anisoplanatic imaging scenario where turbulence warping and blurring vary spatially across the image. The proposed turbulence mitigation (TM) method assumes that a sequence of short-exposure images is acquired. A block matching (BM) registration algorithm is applied to the observed frames for dewarping, and the resulting images are averaged. A convolutional neural network (CNN) is then employed to perform spatially adaptive restoration. We refer to the proposed TM algorithm as the block matching and CNN (BM-CNN) method. Training the CNN is accomplished using simulated data from a fast turbulence simulation tool capable of producing a large amount of degraded imagery from declared truth images rapidly. Testing is done using independent data simulated with a different well-validated numerical wave-propagation simulator. Our proposed BM-CNN TM method is evaluated in a number of experiments using quantitative metrics. The quantitative analysis is made possible by virtue of having truth imagery from the simulations. A number of restored images are provided for subjective evaluation. We demonstrate that the BM-CNN TM method outperforms the benchmark methods in the scenarios tested. © The Authors. Published by SPIE under a Creative Commons Attribution 4.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.

show abstract

Section: Turbulence Characterizationmentioning

confidence: 99%

Section: Turbulence Characterizationmentioning

confidence: 99%

Deep learning for anisoplanatic optical turbulence mitigation in long-range imaging

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…This leads to anisoplanatic conditions where the acquired short-exposure images are corrupted by spatially and temporally varying warp and blur. One simple and effective turbulence mitigation method is the Block Matching and Wiener Filtering (BMWF) algorithm [2,3]. The BMWF method uses a Block Matching Algorithm (BMA) to perform dewarping on a sequence of short exposure frames.…”

Section: Introductionmentioning

confidence: 99%

“…Our OTF model also takes into account how effective the image registration is in performing atmospheric tilt correction. For this, we define and use a parameter that we refer as the tilt correction factor [2,3]. In order to use a turbulence mitigation method such as the BMWF in an automated manner, we need to be able to estimate the Fried parameter from the observed images and separately determine the tilt correction factor.…”

Section: Introductionmentioning

confidence: 99%

Application of tilt correlation statistics to anisoplanatic optical turbulence modeling and mitigation

et al. 2021

Self Cite

View full text Add to dashboard Cite

Atmospheric optical turbulence can be a significant source of image degradation, particularly in long range imaging applications. Many turbulence mitigation algorithms rely on an optical transfer function (OTF) model that includes the Fried parameter. We present anisoplanatic tilt statistics for spherical wave propagation. We transform these into 2D autocorrelation functions that can inform turbulence modeling and mitigation algorithms. Using these, we construct an OTF model that accounts for image registration. We also propose a spectral-ratio Fried parameter estimation algorithm that is robust to camera motion and requires no specialized scene content or sources. We employ the Fried parameter estimation and OTF model for turbulence mitigation. A numerical wave-propagation turbulence simulator is used to generate data to quantitatively validate the proposed methods. Results with real camera data are also presented.

show abstract

“…The second being to use post-acquisition processing, which aims to recover the lost high frequency details of the captured image. The field of turbulence mitigation in imagery is a well researched field [2][3][4][5][6][7][8] , where common techniques make use of temporally varying video sequences to obtain either a single high resolution image or a sequence of clean images.…”

Section: Introductionmentioning

confidence: 99%

Analysis of deep learning architectures for turbulence mitigation in long-range imagery

Vint

Caterina

Soraghan

et al. 2020

Artificial Intelligence and Machine Learning in Defense Applications II

View full text Add to dashboard Cite

In long range imagery, the atmosphere along the line of sight can result in unwanted visual effects. Random variations in the refractive index of the air causes light to shift and distort. When captured by a camera, this randomly induced variation results in blurred and spatially distorted images. The removal of such effects is greatly desired. Many traditional methods are able to reduce the effects of turbulence within images, however they require complex optimisation procedures or have large computational complexity. The use of deep learning for image processing has now become commonplace, with neural networks being able to outperform traditional methods in many fields. This paper presents an evaluation of various deep learning architectures on the task of turbulence mitigation. The core disadvantage of deep learning is the dependence on a large quantity of relevant data. For the task of turbulence mitigation, real life data is difficult to obtain, as a clean undistorted image is not always obtainable. Turbulent images were therefore generated with the use of a turbulence simulator. This was able to accurately represent atmospheric conditions and apply the resulting spatial distortions onto clean images. This paper provides a comparison between current state of the art image reconstruction convolutional neural networks. Each network is trained on simulated turbulence data. They are then assessed on a series of test images. It is shown that the networks are unable to provide high quality output images. However, they are shown to be able to reduce the effects of spatial warping within the test images. This paper provides critical analysis into the effectiveness of the application of deep learning. It is shown that deep learning has potential in this field, and can be used to make further improvements in the future.

show abstract

Fusion of interpolated frames superresolution in the presence of atmospheric optical turbulence

Cited by 12 publications

References 29 publications

Deep learning for anisoplanatic optical turbulence mitigation in long-range imaging

Deep learning for anisoplanatic optical turbulence mitigation in long-range imaging

Application of tilt correlation statistics to anisoplanatic optical turbulence modeling and mitigation

Analysis of deep learning architectures for turbulence mitigation in long-range imagery

Contact Info

Product

Resources

About