Generalized MSFA Engineering With Structural and Adaptive Nonlocal Demosaicing

Bian, Liheng; Wang, Yugang; Zhang, Jun

doi:10.1109/tip.2021.3108913

Cited by 16 publications

(8 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…According to [71], the reconstruction results of the BTES and the compressive sensing algorithms based on the binary tree filter pattern are better than the random and regular models. The optimal model for MSFA is adopted to emphasize the benefits of CAMSRIS.…”

Section: B Simulation Results and Analysismentioning

confidence: 99%

Spectral Clustering Super-Resolution Imaging Based on Multispectral Camera Array

Huang

Chen

Wang

et al. 2023

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

Although multispectral and hyperspectral imaging acquisitions are applied in numerous fields, the existing spectral imaging systems suffer from either low temporal or spatial resolution. In this study, a new multispectral imaging systemcamera array based multispectral super resolution imaging system (CAMSRIS) is proposed that can simultaneously achieve multispectral imaging with high temporal and spatial resolutions. The proposed registration algorithm is used to align pairs of different peripheral and central view images. A novel, superresolution, spectral-clustering-based image reconstruction algorithm was developed for the proposed CAMSRIS to improve the spatial resolution of the acquired images and preserve the exact spectral information without introducing false information. The reconstructed results showed that the spatial and spectral quality and operational efficiency of the proposed system are better than those of a multispectral filter array (MSFA) based on different multispectral datasets. The PSNR of the multispectral super-resolution images obtained by the proposed method were respectively higher by 2.03 and 1.93 dB than those of GAP-TV and DeSCI, and the execution time was significantly shortened by approximately 54.55 s and 9820.19 s when the CAMSI dataset was used. The feasibility of the proposed system was verified in practical applications based on different scenes captured by the self-built system.

show abstract

Section: B Simulation Results and Analysismentioning

confidence: 99%

Spectral Clustering Super-Resolution Imaging Based on Multispectral Camera Array

Huang

Chen

Wang

et al. 2023

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

show abstract

“…The advance in CFA design for RGB cameras sparks great interests in the CFA design for MS cameras [55,45,43,44,1,24,38,47,15,6,63,56]. An early work by Ramanath et al presented a CFA pattern which is composed of seven bands and they are arranged hexagonally [55].…”

Section: Related Workmentioning

confidence: 99%

“…Demosaicing. Demosaicing has been a well-established field with many great works proposed [23,29,35,47,15,6,5]. The general idea is to use the measured signals at sparse locations to fill up the missing values in neighboring locations.…”

Section: Related Workmentioning

confidence: 99%

Jointly Learning Band Selection and Filter Array Design for Hyperspectral Imaging

Dai²,

Gool

2023

2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

A single-shot multispectral camera equipped with an optimized color filter array (CFA) has the potential to deliver a fast and low-cost hyperspectral (HS) imaging system. Previous solutions are largely restricted to designing demosaicing algorithms for fixed CFAs -be it the Bayer color pattern or evenly-spaced spectral multiplexing patterns. Since sampling and reconstruction are tightly-coupled, the ability to search for an optimal solution is severely constrained by using predefined CFAs. In this work, we simultaneously address the problem of spectral band selection, CFA design, image demosaicing, and spectral image recovery in a joint learning framework for single-shot HS imaging. We propose a reinforcement learning (RL) based method for spectral band selection and a novel neural network for CFA generation, image demosaicing, and HS image recovery. The final spectral reconstruction accuracy is used to supervise the training of the main network to maximize the synergies between those tightly-related tasks. The RL method regards the main network as an agent to collect reward. Our final method delivers a simple setup -as simple as an RGB camera -for HS imaging. Experimental results show that our method outperforms competing methods by a large margin.

show abstract

“…5 a). Bian et al [44] experimentally proved that NLR algorithms with multi-channel block matching can achieve state-of-the-art performance for multi-spectral demosaicing. Inheriting Bian et al's framework, GLR replaces the matching strategy with multi-channel global matching (see Supplement 1).…”

Section: Multi-spectral Filter Array Demosaicingmentioning

confidence: 99%

Large-scale Global Low-rank Optimization for Computational Compressed Imaging

Li¹,

Xu²,

Chen³

et al. 2023

Preprint

View full text Add to dashboard Cite

Computational reconstruction plays a vital role in computer vision and computational photography. Most of the conventional optimization and deep learning techniques explore local information for reconstruction. Recently, nonlocal low-rank (NLR) reconstruction has achieved remarkable success in improving accuracy and generalization. However, the computational cost has inhibited NLR from seeking global structural similarity, which consequentially keeps it trapped in the tradeoff between accuracy and efficiency and prevents it from high-dimensional large-scale tasks. To address this challenge, we report here the global low-rank (GLR) optimization technique, realizing highly-efficient large-scale reconstruction with global self-similarity. Inspired by the self-attention mechanism in deep learning, GLR extracts exemplar image patches by feature detection instead of conventional uniform selection. This directly produces key patches using structural features to avoid burdensome computational redundancy. Further, it performs patch matching across the entire image via neural-based convolution, which produces the global similarity heat map in parallel, rather than conventional sequential block-wise matching. As such, GLR improves patch grouping efficiency by more than one order of magnitude. We experimentally demonstrate GLR's effectiveness on temporal, frequency, and spectral dimensions, including different computational imaging modalities of compressive temporal imaging, magnetic resonance imaging, and multispectral filter array demosaicing. This work presents the superiority of inherent fusion of deep learning strategies and iterative optimization, and breaks the persistent dilemma of the tradeoff between accuracy and efficiency for various large-scale reconstruction tasks.

show abstract

Generalized MSFA Engineering With Structural and Adaptive Nonlocal Demosaicing

Cited by 16 publications

References 54 publications

Spectral Clustering Super-Resolution Imaging Based on Multispectral Camera Array

Spectral Clustering Super-Resolution Imaging Based on Multispectral Camera Array

Jointly Learning Band Selection and Filter Array Design for Hyperspectral Imaging

Large-scale Global Low-rank Optimization for Computational Compressed Imaging

Contact Info

Product

Resources

About