Serial Decoders-Based Auto-Encoders for Image Reconstruction

Li, Honggui; Trocan, Maria; Galayko, Dimitri

doi:10.3390/app12168256

Cited by 2 publications

(1 citation statement)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Inspired by the successful application of the deep network model in computer vision tasks, scholars have also applied the deep network to snapshot compressive imaging tasks [15][16][17][18][19][20][21]. Their main idea is to design a network to directly learn the inverse mapping from the snapshot measurement to the original signal.…”

Section: Introductionmentioning

confidence: 99%

Transformer-Based Cascading Reconstruction Network for Video Snapshot Compressive Imaging

et al. 2023

View full text Add to dashboard Cite

Video Snapshot Compressive Imaging (SCI) is a new imaging method based on compressive sensing. It encodes image sequences into a single snapshot measurement and then recovers the original high-speed video through reconstruction algorithms, which has the advantages of a low hardware cost and high imaging efficiency. How to construct an efficient algorithm is the key problem of video SCI. Although the current mainstream deep convolution network reconstruction methods can directly learn the inverse reconstruction mapping, they still have shortcomings in the representation of the complex spatiotemporal content of video scenes and the modeling of long-range contextual correlation. The quality of reconstruction still needs to be improved. To solve this problem, we propose a Transformer-based Cascading Reconstruction Network for Video Snapshot Compressive Imaging. In terms of the long-range correlation matching in the Transformer, the proposed network can effectively capture the spatiotemporal correlation of video frames for reconstruction. Specifically, according to the residual measurement mechanism, the reconstruction network is configured as a cascade of two stages: overall structure reconstruction and incremental details reconstruction. In the first stage, a multi-scale Transformer module is designed to extract the long-range multi-scale spatiotemporal features and reconstruct the overall structure. The second stage takes the measurement of the first stage as the input and employs a dynamic fusion module to adaptively fuse the output features of the two stages so that the cascading network can effectively represent the content of complex video scenes and reconstruct more incremental details. Experiments on simulation and real datasets show that the proposed method can effectively improve the reconstruction accuracy, and ablation experiments also verify the validity of the constructed network modules.

show abstract

Section: Introductionmentioning

confidence: 99%

Transformer-Based Cascading Reconstruction Network for Video Snapshot Compressive Imaging

et al. 2023

View full text Add to dashboard Cite

show abstract

Switchable-Encoder-Based Self-Supervised Learning Framework for Monocular Depth and Pose Estimation

Kim,

Gao,

Park

et al. 2023

Remote Sensing

View full text Add to dashboard Cite

Monocular depth prediction research is essential for expanding meaning from 2D to 3D. Recent studies have focused on the application of a newly proposed encoder; however, the development within the self-supervised learning framework remains unexplored, an aspect critical for advancing foundational models of 3D semantic interpretation. Addressing the dynamic nature of encoder-based research, especially in performance evaluations for feature extraction and pre-trained models, this research proposes the switchable encoder learning framework (SELF). SELF enhances versatility by enabling the seamless integration of diverse encoders in a self-supervised learning context for depth prediction. This integration is realized through the direct transfer of feature information from the encoder and by standardizing the input structure of the decoder to accommodate various encoder architectures. Furthermore, the framework is extended and incorporated into an adaptable decoder for depth prediction and camera pose learning, employing standard loss functions. Comparative experiments with previous frameworks using the same encoder reveal that SELF achieves a 7% reduction in parameters while enhancing performance. Remarkably, substituting newly proposed algorithms in place of an encoder improves the outcomes as well as significantly decreases the number of parameters by 23%. The experimental findings highlight the ability of SELF to broaden depth factors, such as depth consistency. This framework facilitates the objective selection of algorithms as a backbone for extended research in monocular depth prediction.

show abstract

Serial Decoders-Based Auto-Encoders for Image Reconstruction

Cited by 2 publications

References 33 publications

Transformer-Based Cascading Reconstruction Network for Video Snapshot Compressive Imaging

Transformer-Based Cascading Reconstruction Network for Video Snapshot Compressive Imaging

Switchable-Encoder-Based Self-Supervised Learning Framework for Monocular Depth and Pose Estimation

Contact Info

Product

Resources

About