Multispectral single-sensor RGB-NIR imaging: New challenges and opportunities

Soria, Xavier Gimeno; Sappa, Ángel D.; Akbarinia, Arash

doi:10.1109/ipta.2017.8310105

Cited by 23 publications

(17 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The effectiveness of the different approaches was assessed using a Single Sensor Multi-Spectral Image Dataset (SSMID) [ 16 ]. In particular, the Outdoor Multi-Spectral Image with Vegetation (OMSIV) was used.…”

Section: Resultsmentioning

confidence: 99%

“…Then, a pair of acquired images was registered [ 41 ]. For a more detailed description of SSIMD, see [ 16 ]. Since there is a high amount of band cross-talk in images, a big volume of training data was required to train the proposed models.…”

Section: Resultsmentioning

confidence: 99%

“…

and

are the factors to be adjusted according to different viewing parameters; the

function is intended to improve the color difference equation to fit chromatic differences in the blue region (for a detailed description see [ 44 , 45 ]). Once

was computed for the whole set of pairs of pixels, the

value was obtained as presented in [ 16 ]. It should be mentioned that

represents the percentage of pixels with a

value in between two given thresholds [a, b].…”

Section: Resultsmentioning

confidence: 99%

“…In [ 20 ] the authors presented three methods which were evaluated with different demosaicing algorithms and linear mappings. In the current work, following the suggestion from [ 20 ] and according to the type of sensor used in our experiments [ 16 ], a uniform RGB-NIR pattern [ 11 ] was used, then demosaicing was performed using [ 13 ], and finally, color correction was obtained through polynomial linear mapping [ 46 ], since this combination produces the best performance. The second algorithm used to compare the proposed approach was presented in [ 21 ].…”

Section: Resultsmentioning

confidence: 99%

“…The codes of these fully trained networks are available through . To extend our previous work [ 15 ] via reporting additional results and findings on a publicly available dataset [ 16 ]. This dataset has been enlarged through corresponding data augmentation.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Wide-Band Color Imagery Restoration for RGB-NIR Single Sensor Images

Soria

Sappa

Hammoud

2018

Sensors

Self Cite

View full text Add to dashboard Cite

Multi-spectral RGB-NIR sensors have become ubiquitous in recent years. These sensors allow the visible and near-infrared spectral bands of a given scene to be captured at the same time. With such cameras, the acquired imagery has a compromised RGB color representation due to near-infrared bands (700–1100 nm) cross-talking with the visible bands (400–700 nm). This paper proposes two deep learning-based architectures to recover the full RGB color images, thus removing the NIR information from the visible bands. The proposed approaches directly restore the high-resolution RGB image by means of convolutional neural networks. They are evaluated with several outdoor images; both architectures reach a similar performance when evaluated in different scenarios and using different similarity metrics. Both of them improve the state of the art approaches.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

“…

and

are the factors to be adjusted according to different viewing parameters; the

function is intended to improve the color difference equation to fit chromatic differences in the blue region (for a detailed description see [ 44 , 45 ]). Once

was computed for the whole set of pairs of pixels, the

value was obtained as presented in [ 16 ]. It should be mentioned that

represents the percentage of pixels with a

value in between two given thresholds [a, b].…”

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Wide-Band Color Imagery Restoration for RGB-NIR Single Sensor Images

Soria

Sappa

Hammoud

2018

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

Monolithic RGB–NIR Perovskite Photodetector for Fused Multispectral Sensing and Imaging

Liu

et al. 2022

J. Phys. Chem. Lett.

View full text Add to dashboard Cite

The multispectral fusion of near-infrared (NIR) and visible red−green−blue (RGB) photons can enhance target identification under weak light conditions. Nevertheless, the crosstalk between NIR and RGB photons in a traditional pixelated sensor impedes their practical application, while using complex algorithms and optical filters would significantly increase the cost, form factor, and frame latency. In this work, a delicate monolithic RGBN (RGB−NIR) multispectral photodetector (PD) is proposed on the basis of perovskite materials without complicated algorithms or optical filters. The multispectral response toward selective RGBN signals in this monolithic PD pixels can be achieved by switching the polarity of the applied bias, affording the following benefits: I on /I off ratio of >10 4 , detectivity of >10 10 Jones, crosstalk of −74 dB, and fast response with −3 dB > 10 3 Hz. Moreover, proof-of-concept imaging of the iris and periocular with successful recognition in multispectral fusion further confirms its potential for identity authentication.

show abstract

Multi-Spectral RGB-NIR Image Classification Using Double-Channel CNN

et al. 2019

View full text Add to dashboard Cite

As 4-sensor line scan camera technology has matured, red (R), green (G), blue (B), and near-infrared (RGB-NIR) datasets have begun to appear in large numbers. The RGB-NIR data contain the rich color features of the RGB image and the sharp edge features of the NIR image. At present, in many studies, the RGB-NIR data are input directly into the processing algorithms for calculation of the 4D data; in these cases, redundant information is included, and the high correlation between the bands results in an inability to fully exploit the characteristics of the RGB-NIR data. In this paper, we propose a double-channel convolutional neural network (CNN) algorithm that takes into account the strong correlation between the R, G, and B bands in aerial images and the weaker correlation between the NIR band and the R, G, and B bands. First, the features of the RGB and NIR bands are calculated in two different CNN networks, and subsequently, feature fusion is performed in the fully connected layer. This is followed by the classification. By combining the two neural networks of RGB-CNN and NIR-CNN, the respective characteristics of the RGB-NIR data are fully exploited. INDEX TERMS Multi-spectral, CNN, RGB-NIR, double-channel CNN.

show abstract

Multispectral single-sensor RGB-NIR imaging: New challenges and opportunities

Cited by 23 publications

References 20 publications

Wide-Band Color Imagery Restoration for RGB-NIR Single Sensor Images

Wide-Band Color Imagery Restoration for RGB-NIR Single Sensor Images

Monolithic RGB–NIR Perovskite Photodetector for Fused Multispectral Sensing and Imaging

Multi-Spectral RGB-NIR Image Classification Using Double-Channel CNN

Contact Info

Product

Resources

About