BIRNAT: Bidirectional Recurrent Neural Networks with Adversarial Training for Video Snapshot Compressive Imaging

Cheng, Ziheng; Lu, Ruiying; Wang, Zhengjue; Zhang, Hao; Chen, Bo; Meng, Ziyi; Yuan, Xin

doi:10.1007/978-3-030-58586-0_16

Cited by 67 publications

(81 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This demonstrates For our learned network, we have experimented with various network architectures. Specifically, for the DE-RNN model, we adopt the architecture from BIRNAT [7]. Regarding its two-stage (forward+backward) RNN as a whole, we iteratively feed the output of the backward RNN back as the input of the forward one.…”

Section: Convergence Theorymentioning

confidence: 99%

“…where y ∈ R n is the 2D measurement with n equaling the number of each video frame's pixels, Φ ∈ R n×nB is the We test our model under two different frameworks, i.e., RNN [7] and PnP-GAP [34], the fidelity and stability of our model can be obviously observed.…”

Section: Introductionmentioning

confidence: 99%

“…The rapid advancement of deep learning and artificial intelligence have empowered a new wave of revolutionary solutions towards these previously intractable problems. For instance, BIRNAT [7] employed recurrent neural networks (RNNs) to reconstruct the video frames in a sequential manner and explore the temporal correlation within the video SCI signal. Inspired by particular optimization algorithms, GAP-net [16], DUN-3DUnet [28] designed deep unfolding structures, which consist of a fixed number of architecturally identical blocks.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Deep Equilibrium Models for Video Snapshot Compressive Imaging

Zhao¹,

Yuan²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

The ability of snapshot compressive imaging (SCI) systems to efficiently capture high-dimensional (HD) data has led to an inverse problem, which consists of recovering the HD signal from the compressed and noisy measurement. While reconstruction algorithms grow fast to solve it with the recent advances of deep learning, the fundamental issue of accurate and stable recovery remains. To this end, we propose deep equilibrium models (DEQ) for video SCI, fusing data-driven regularization and stable convergence in a theoretically sound manner. Each equilibrium model implicitly learns a nonexpansive operator and analytically computes the fixed point, thus enabling unlimited iterative steps and infinite network depth with only a constant memory requirement in training and testing. Specifically, we demonstrate how DEQ can be applied to two existing models for video SCI reconstruction: recurrent neural networks (RNN) and Plug-and-Play (PnP) algorithms. On a variety of datasets and real data, both quantitative and qualitative evaluations of our results demonstrate the effectiveness and stability of our proposed method. The code and models will be released to the public.

show abstract

Section: Convergence Theorymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep Equilibrium Models for Video Snapshot Compressive Imaging

Zhao¹,

Yuan²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…As shown in Figure 1, we use a CNN and two RNNs to perform the reconstruction. 51,52 The more information RNN takes as input, the better results we get. Therefore, the first frame for RNN with as much visual information as possible is required.…”

Section: Articlementioning

confidence: 99%

Super-compression of large electron microscopy time series by deep compressive sensing learning

Zheng

Wang

Yuan³

et al. 2021

Patterns

Self Cite

View full text Add to dashboard Cite

show abstract

“…These methods of learning measurement matrices can obtain matrices that are more suitable for natural images, but the training process for these matrices is complicated and at the same time highly dependent on the training set. In recent years, DNN has also been applied to CS video recovery [ 21 , 22 ].…”

Section: Basic Concepts and Related Workmentioning

confidence: 99%

Dual Optical Path Based Adaptive Compressive Sensing Imaging System

Lü

Xue

et al. 2021

Sensors

View full text Add to dashboard Cite

Compressive Sensing (CS) has proved to be an effective theory in the field of image acquisition. However, in order to distinguish the difference between the measurement matrices, the CS imaging system needs to have a higher signal sampling accuracy. At the same time, affected by the noise of the light path and the circuit, the measurements finally obtained are noisy, which directly affects the imaging quality. We propose a dual-optical imaging system that uses the bidirectional reflection characteristics of digital micromirror devices (DMD) to simultaneously acquire CS measurements and images under the same viewing angle. Since deep neural networks have powerful modeling capabilities, we trained the filter network and the reconstruction network separately. The filter network is used to filter the noise in the measurements, and the reconstruction network is used to reconstruct the CS image. Experiments have proved that the method we proposed can filter the noise in the sampling process of the CS system, and can significantly improve the quality of image reconstruction under a variety of algorithms.

show abstract

BIRNAT: Bidirectional Recurrent Neural Networks with Adversarial Training for Video Snapshot Compressive Imaging

Cited by 67 publications

References 48 publications

Deep Equilibrium Models for Video Snapshot Compressive Imaging

Deep Equilibrium Models for Video Snapshot Compressive Imaging

Super-compression of large electron microscopy time series by deep compressive sensing learning

Dual Optical Path Based Adaptive Compressive Sensing Imaging System

Contact Info

Product

Resources

About