HSCNN+: Advanced CNN-Based Hyperspectral Recovery from RGB Images

Shi, Zhan; Chang, Chen; Xiong, Zhiwei; Liu, Dong; Wu, Feng

doi:10.1109/cvprw.2018.00139

Cited by 221 publications

(214 citation statements)

References 30 publications

Supporting

Mentioning

211

Contrasting

Order By: Relevance

“…We build our models based on the HSCNN-R architecture, which is the 2nd place entry of the NTIRE2018 [5], [35] (whose performance is similar to the 1st place HSCNN-D model; we use the 2nd place architecture simply because it was simpler in our development environment). As illustrated in Figure 4, the HSCNN-R model adopts a deep residual learning framework [21].…”

Section: Methodsmentioning

confidence: 99%

“…The Fig. 4: The HSCNN-R architecture [35]. 'C' means 3 × 3 convolution and 'R' refers to ReLU activation.…”

Section: Methodsmentioning

confidence: 99%

“…Indeed, it is well known that faces, chlorophyl (in foliage) and daylights have very characteristic shapes (amongst other scene features). Of the current developments, deep neural networks [3], [24], [36], [15], [35] provide the leading performance in spectral reconstruction.…”

Section: Related Workmentioning

confidence: 99%

“…Spectral reconstruction (SR) is an alternative approach to recording hyperspectral information, where hyperspectral images are recovered from RGB images [30], [3], [26], [24], [36], [15], [7], [26], [22], [4], [1], [15], [35], [5]. The idea is not as naïve as it might first appear.…”

Section: Introductionmentioning

confidence: 99%

“…• We evaluate the state-of-the-art HSCNN-D and HSCNN-R models [5], [35] to gauge the extent that they deliver physically plausible spectral recovery, either in the sense of predicting the input RGBs or being resilient to a varying exposure.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Exposure Invariance in Spectral Reconstruction from RGB Images

Lin

Finlayson

2019

cic

View full text Add to dashboard Cite

Recently Convolutional Neural Networks (CNN) have been used to reconstruct hyperspectral information from RGB images. Moreover, this spectral reconstruction problem (SR) can often be solved with good (low) error. However, these methods are not physically plausible: that is when the recovered spectra are reintegrated with the underlying camera sensitivities, the resulting predicted RGB is not the same as the actual RGB, and sometimes this discrepancy can be large. The problem is further compounded by exposure change. Indeed, most learningbased SR models train for a fixed exposure setting and we show that this can result in poor performance when exposure varies.In this paper we show how CNN learning can be extended so that physical plausibility is enforced and the problem resulting from changing exposures is mitigated. Our SR solution improves the state-of-the-art spectral recovery performance under varying exposure conditions while simultaneously ensuring physical plausibility (i.e. the recovered spectra reintegrate to the input RGBs exactly).

show abstract

Section: Methodsmentioning

confidence: 99%

“…The Fig. 4: The HSCNN-R architecture [35]. 'C' means 3 × 3 convolution and 'R' refers to ReLU activation.…”

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Exposure Invariance in Spectral Reconstruction from RGB Images

Lin

Finlayson

2019

cic

View full text Add to dashboard Cite

show abstract

An investigation on worst‐case spectral reconstruction from RGB images via Radiance Mondrian World assumption

Lin

Finlayson

2023

Color Research & Application

View full text Add to dashboard Cite

Spectral reconstruction (SR) algorithms recover hyperspectral measurements from RGB camera responses. Statistical models at different levels of complexity are used to solve the SR problem-from the simplest closed-form regression, to sparse coding, to the complex deep neural networks (DNN). Recently, these methods were benchmarked based on the mean performance of the models and on a fixed set of real-world scenes, suggesting that more complex (more non-linear) models generally deliver better SR. In this paper, we investigate the relative performances of these models in terms of a real-world worst-case imaging condition called the Radiance Mondrian World (RMW) assumption. Under the RMW, testing hyperspectral images are composed of randomlyarranged and overlapping rectangular patches, where each patch is filled with one random radiance spectrum uniformly sampled from the convex closure of all natural radiances (i.e., all spectra in the concerned hyperspectral image dataset). Surprisingly, we show that all compared algorithms-regardless of their model complexity-degrade to broadly the same level of performance on our RMW testing set. Further, by retraining all models with an RMW training set, we show that increasing model complexity also does not help learning better SR mappings from the RMW images. That is, using simple regression is as good as using a DNN. This similarity of performance is also shown to hold for images adhering to the conventional Mondrian World assumption (random reflectances lit by a single, per scene, randomly selected light source).

show abstract