2D-3D CNN Based Architectures for Spectral Reconstruction from RGB Images

Koundinya, Sriharsha; Sharma, Himanshu; Sharma, Manoj Kumar; Upadhyay, Avinash; Manekar, Raunak; Mukhopadhyay, Rudrabha; Karmakar, Arindam; Chaudhury, Santanu

doi:10.1109/cvprw.2018.00129

Cited by 60 publications

(29 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As we focused on RGB based spatial resolution increasing of multispectral data and the use of a common dataset for testing, the CAVE dataset was the chosen dataset. This dataset was also used for hyperspectral recovery, unmixing, and of course upsampling, and Super-Resolution in the works of [ 22 , 25 , 31 , 34 , 63 ]. Below, we will make some considerations between our results and the results found in these previous works.…”

Section: Discussionmentioning

confidence: 99%

“…The CAVE dataset [ 45 ] is a common [ 25 , 31 , 34 , 37 , 50 ] dataset for spectral reconstruction and/or Super-Resolution validation. It consists of 32 varied scenes grouped in “stuff”, “skin and hair”, “paints”, “food and drinks”, and “real and fake” images.…”

Section: Methodsmentioning

confidence: 99%

“…The advances in computational power and a great interest to resolve the high-resolution problems with Super-Resolution (SR) networks based on Convolutional Neural Networks (CNN) have arisen from the work of [ 30 ], which inspired works to improve the spatial resolution of spectral images [ 19 ] and image resolution in general [ 23 , 31 , 32 , 33 , 34 , 35 ], also influenced by the NTIRE [ 36 ] and PIRM2018 [ 37 ] contests. Most of them rely on benchmark datasets or specific datasets available only for the study.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Improving Spatial Resolution of Multispectral Rock Outcrop Images Using RGB Data and Artificial Neural Networks

Marques

Souza

Müller

et al. 2020

Sensors

View full text Add to dashboard Cite

Spectral information provided by multispectral and hyperspectral sensors has a great impact on remote sensing studies, easing the identification of carbonate outcrops that contribute to a better understanding of petroleum reservoirs. Sensors aboard satellites like Landsat series, which have data freely available usually lack the spatial resolution that suborbital sensors have. Many techniques have been developed to improve spatial resolution through data fusion. However, most of them have serious limitations regarding application and scale. Recently Super-Resolution (SR) convolution neural networks have been tested with encouraging results. However, they require large datasets, more time and computational power for training. To overcome these limitations, this work aims to increase the spatial resolution of multispectral bands from the Landsat satellite database using a modified artificial neural network that uses pixel kernels of a single spatial high-resolution RGB image from Google Earth as input. The methodology was validated with a common dataset of indoor images as well as a specific area of Landsat 8. Different downsized scale inputs were used for training where the validation used the ground truth of the original size images, obtaining comparable results to the recent works. With the method validated, we generated high spatial resolution spectral bands based on RGB images from Google Earth on a carbonated outcrop area, which were then properly classified according to the soil spectral responses making use of the advantage of a higher spatial resolution dataset.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Improving Spatial Resolution of Multispectral Rock Outcrop Images Using RGB Data and Artificial Neural Networks

Marques

Souza

Müller

et al. 2020

Sensors

View full text Add to dashboard Cite

show abstract

“…Therefore the reconstructed spectral accuracy is low, the speed is slow, and the running cost is high. Other supervised methods [ 35 ] exist such as the convolution neural network (CNN) [ 36 ]. In this method, the two-dimensional CNN model mainly focuses on extracting spectral data by only considering spatial correlation.…”

Section: Introductionmentioning

confidence: 99%

Double Ghost Convolution Attention Mechanism Network: A Framework for Hyperspectral Reconstruction of a Single RGB Image

Wang

Jiangwei

2021

Sensors

View full text Add to dashboard Cite

Current research on the reconstruction of hyperspectral images from RGB images using deep learning mainly focuses on learning complex mappings through deeper and wider convolutional neural networks (CNNs). However, the reconstruction accuracy of the hyperspectral image is not high and among other issues the model for generating these images takes up too much storage space. In this study, we propose the double ghost convolution attention mechanism network (DGCAMN) framework for the reconstruction of a single RGB image to improve the accuracy of spectral reconstruction and reduce the storage occupied by the model. The proposed DGCAMN consists of a double ghost residual attention block (DGRAB) module and optimal nonlocal block (ONB). DGRAB module uses GhostNet and PRELU activation functions to reduce the calculation parameters of the data and reduce the storage size of the generative model. At the same time, the proposed double output feature Convolutional Block Attention Module (DOFCBAM) is used to capture the texture details on the feature map to maximize the content of the reconstructed hyperspectral image. In the proposed ONB, the Argmax activation function is used to obtain the region with the most abundant feature information and maximize the most useful feature parameters. This helps to improve the accuracy of spectral reconstruction. These contributions enable the DGCAMN framework to achieve the highest spectral accuracy with minimal storage consumption. The proposed method has been applied to the NTIRE 2020 dataset. Experimental results show that the proposed DGCAMN method outperforms the spectral accuracy reconstructed by advanced deep learning methods and greatly reduces storage consumption.

show abstract

“…Spectral reconstruction (SR) is an alternative approach to recording hyperspectral information, where hyperspectral images are recovered from RGB images [30], [3], [26], [24], [36], [15], [7], [26], [22], [4], [1], [15], [35], [5]. The idea is not as naïve as it might first appear.…”

Section: Introductionmentioning

confidence: 99%

Exposure Invariance in Spectral Reconstruction from RGB Images

Lin

Finlayson

2019

cic

View full text Add to dashboard Cite

Recently Convolutional Neural Networks (CNN) have been used to reconstruct hyperspectral information from RGB images. Moreover, this spectral reconstruction problem (SR) can often be solved with good (low) error. However, these methods are not physically plausible: that is when the recovered spectra are reintegrated with the underlying camera sensitivities, the resulting predicted RGB is not the same as the actual RGB, and sometimes this discrepancy can be large. The problem is further compounded by exposure change. Indeed, most learningbased SR models train for a fixed exposure setting and we show that this can result in poor performance when exposure varies.In this paper we show how CNN learning can be extended so that physical plausibility is enforced and the problem resulting from changing exposures is mitigated. Our SR solution improves the state-of-the-art spectral recovery performance under varying exposure conditions while simultaneously ensuring physical plausibility (i.e. the recovered spectra reintegrate to the input RGBs exactly).

show abstract

2D-3D CNN Based Architectures for Spectral Reconstruction from RGB Images

Cited by 60 publications

References 38 publications

Improving Spatial Resolution of Multispectral Rock Outcrop Images Using RGB Data and Artificial Neural Networks

Improving Spatial Resolution of Multispectral Rock Outcrop Images Using RGB Data and Artificial Neural Networks

Double Ghost Convolution Attention Mechanism Network: A Framework for Hyperspectral Reconstruction of a Single RGB Image

Exposure Invariance in Spectral Reconstruction from RGB Images

Contact Info

Product

Resources

About