Learning Cross-Spectral Similarity Measures with Deep Convolutional Neural Networks

Aguilera, Cristhian; Aguilera, Francisco; Sappa, Ángel D.; Toledo, Ricardo

doi:10.1109/cvprw.2016.40

Cited by 72 publications

(76 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For deep learning methods, Aguilera et al (Aguilera et al 2016) learned a similarity measurement of cross-spectral image patches, which is a potential way to learn matching cost for multi-spectrum images. Zhi et al (Zhi et al 2018) utilized deep segmentation maps to improve robustness of cross-spectral stereo matching, while the method required extra semantic annotations and manually designed losses for different materials, which made it hard to apply to other scenes.…”

Section: Cross-spectral Stereo Matchingmentioning

confidence: 99%

Unsupervised Cross-Spectral Stereo Matching by Learning to Synthesize

Liang

Guo

et al. 2019

AAAI

View full text Add to dashboard Cite

Unsupervised cross-spectral stereo matching aims at recovering disparity given cross-spectral image pairs without any depth or disparity supervision. The estimated depth provides additional information complementary to original images, which can be helpful for other vision tasks such as tracking, recognition and detection. However, there are large appearance variations between images from different spectral bands, which is a challenge for cross-spectral stereo matching. Existing deep unsupervised stereo matching methods are sensitive to the appearance variations and do not perform well on cross-spectral data. We propose a novel unsupervised crossspectral stereo matching framework based on image-to-image translation. First, a style adaptation network transforms images across different spectral bands by cycle consistency and adversarial learning, during which appearance variations are minimized. Then, a stereo matching network is trained with image pairs from the same spectra using view reconstruction loss. At last, the estimated disparity is utilized to supervise the spectral translation network in an end-to-end way. Moreover, a novel style adaptation network F-cycleGAN is proposed to improve the robustness of spectral translation. Our method can tackle appearance variations and enhance the robustness of unsupervised cross-spectral stereo matching. Experimental results show that our method achieves good performance without using depth supervision or explicit semantic information.

show abstract

Section: Cross-spectral Stereo Matchingmentioning

confidence: 99%

Unsupervised Cross-Spectral Stereo Matching by Learning to Synthesize

Liang

Guo

et al. 2019

AAAI

View full text Add to dashboard Cite

show abstract

“…The dataset used in [7] has been considered in the current work to train and validate the proposed network. This dataset has been obtained from [8], and consists of more than 1 million VIS-NIR cross-spectral image pairs divided into nine different categories.…”

Section: Datasetmentioning

confidence: 99%

“…In a more recent work, [7] tested different CNN-based networks to measure the similarity between images from the VIS-NIR and the VIS-LWIR spectra. In their experiments, they showed that CNN-based networks can outperform the state-of-the-art in terms of matching performance.…”

Section: Cross-spectral Descriptorsmentioning

confidence: 99%

“…We propose three simple and naive solutions: i) y is an RGB non-matching image, ii) y is an NIR non-matching image and iii) y is randomly chosen between RGB and NIR. We test each one of the aforementioned solutions in the dataset used in [7] and presented in section 2 . We train each network nine times, once per category and tested on the other eight categories.…”

Section: Cross-spectral Pn-netmentioning

confidence: 99%

“…FPR95 performance on the VIS-NIR scene dataset. Each network, i.e., siamese-L2, PN-Net and Q-Net, were trained in the country sequence and tested in the other eight sequences as in [7]. Smaller results indicate better performance.…”

Section: Multi-view Stereo Correspondence Datasetmentioning

confidence: 99%

See 2 more Smart Citations

Cross-Spectral Local Descriptors via Quadruplet Network

Aguilera

Sappa

Toledo

2017

Preprint

Self Cite

View full text Add to dashboard Cite

This paper presents a novel CNN-based architecture, referred to as Q-Net, to learn local feature descriptors that are useful for matching image patches from two different spectral bands. Given correctly matched and non-matching cross-spectral image pairs, a quadruplet network is trained to map input image patches to a common Euclidean space, regardless of the input spectral band. Our approach is inspired by the recent success of triplet networks in the visible spectrum, but adapted for cross-spectral scenarios, where for each matching pair there are always two possible non-matching patches; one for each spectrum. Experimental evaluations on a public cross-spectral VIS-NIR dataset shows that the proposed approach improves the state-of-the-art. Moreover, the proposed technique can also be used in mono-spectral settings, obtaining a similar performance to triplet network descriptors, but requiring less training data.

show abstract

Conventional Vision or Not: A Selection of Low‐level Algorithms

2021

From AI to Autonomous and Connected Vehicles

View full text Add to dashboard Cite

Learning Cross-Spectral Similarity Measures with Deep Convolutional Neural Networks

Cited by 72 publications

References 15 publications

Unsupervised Cross-Spectral Stereo Matching by Learning to Synthesize

Unsupervised Cross-Spectral Stereo Matching by Learning to Synthesize

Cross-Spectral Local Descriptors via Quadruplet Network

Conventional Vision or Not: A Selection of Low‐level Algorithms

Contact Info

Product

Resources

About