Subsampled turbulence removal network

Chak, Wai Ho; Lau, Chun Pong; Lui, Lok Ming

doi:10.4310/mcgd.2020.v1.n1.a1

Cited by 2 publications

(3 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Turbulence simulation in the context of deep learning has been reported in [36,23,14]. Their model generates the geometric distortions by repeatedly smoothing a set of random spikes, and the blur is assumed to be spatially invariant Gaussian [13]. We argue that for the face images studied in [36,23,14], the narrow field of view makes their simplified model valid.…”

Section: Training Data: Synthetic Data Generating Schemementioning

confidence: 92%

See 1 more Smart Citation

Single Frame Atmospheric Turbulence Mitigation: A Benchmark Study and A New Physics-Inspired Transformer Model

Mao¹,

Jaiswal²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Image restoration algorithms for atmospheric turbulence are known to be much more challenging to design than traditional ones such as blur or noise because the distortion caused by the turbulence is an entanglement of spatially varying blur, geometric distortion, and sensor noise. Existing CNN-based restoration methods built upon convolutional kernels with static weights are insufficient to handle the spatially dynamical atmospheric turbulence effect. To address this problem, in this paper, we propose a physics-inspired transformer model for imaging through atmospheric turbulence. The proposed network utilizes the power of transformer blocks to jointly extract a dynamical turbulence distortion map and restore a turbulence-free image. In addition, recognizing the lack of a comprehensive dataset, we collect and present two new real-world turbulence datasets that allow for evaluation with both classical objective metrics (e.g., PSNR and SSIM) and a new task-driven metric using text recognition accuracy. The code and datasets are available at github.com/VITA-Group/TurbNet.

show abstract

Section: Training Data: Synthetic Data Generating Schemementioning

confidence: 92%

“…Existing algorithms are all trained with synthetic data. The computationally least expensive synthesis technique is based on the random pixel displacement + blur model [15,13]. In the optics community, there are techniques based on ray-tracing and wave-propagation [25,27,7].…”

Section: Related Workmentioning

confidence: 99%

Single Frame Atmospheric Turbulence Mitigation: A Benchmark Study and A New Physics-Inspired Transformer Model

Mao¹,

Jaiswal²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Subsequent work of Lou et al [20] followed the blur-then-tilt model, and this tradition continues until today where many of the latest deep learning based approaches also use the blurthen-tilt model. For example, Lau et al [21], Nair and Patel [22], Yasarla and Patel [23], and Lau and Lui [24], [25] all explicitly or implicitly use the blur-then-tilt model.…”

Section: Introductionmentioning

confidence: 99%

Tilt-then-Blur or Blur-then-Tilt? Clarifying the Atmospheric Turbulence Model

Chan¹

2022

Preprint

View full text Add to dashboard Cite

Imaging at a long distance often requires advanced image restoration algorithms to compensate for the distortions caused by atmospheric turbulence. However, unlike many standard restoration problems such as deconvolution, the forward image formation model of the atmospheric turbulence does not have a simple expression. Thanks to the Zernike representation of the phase, one can show that the forward model is a combination of tilt (pixel shifting due to the linear phase terms) and blur (image smoothing due to the high order aberrations).Confusions then arise between the ordering of the two operators. Should the model be tilt-then-blur, or blur-then-tilt? Some papers in the literature say that the model is tilt-thenblur, whereas more papers say that it is blur-then-tilt. This paper clarifies the differences between the two and discusses why the tilt-then-blur is the correct model. Recommendations are given to the research community.

show abstract

Subsampled turbulence removal network

Cited by 2 publications

References 14 publications

Single Frame Atmospheric Turbulence Mitigation: A Benchmark Study and A New Physics-Inspired Transformer Model

Single Frame Atmospheric Turbulence Mitigation: A Benchmark Study and A New Physics-Inspired Transformer Model

Tilt-then-Blur or Blur-then-Tilt? Clarifying the Atmospheric Turbulence Model

Contact Info

Product

Resources

About