Learning Product Codebooks Using Vector-Quantized Autoencoders for Image Retrieval

Wu, Hanti; Fieri, Markus

doi:10.1109/globalsip45357.2019.8969272

Cited by 14 publications

(6 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A fine trained model can encode useful information from images into latent variable

and will have a non-zero KL divergence term and a relatively small reconstruction term. However, the straightforward training of VAE can suffer from posterior collapse [ 45 , 50 ], and it fails to make use of enough information. When the posterior collapse phenomenon occurs, the model ends up relying solely on the auto-regressive properties of the decoder while ignoring the latent variables, which become uninformative.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

A Visual and VAE Based Hierarchical Indoor Localization Method

Jiang

Zou

Chen

et al. 2021

Sensors

View full text Add to dashboard Cite

Precise localization and pose estimation in indoor environments are commonly employed in a wide range of applications, including robotics, augmented reality, and navigation and positioning services. Such applications can be solved via visual-based localization using a pre-built 3D model. The increase in searching space associated with large scenes can be overcome by retrieving images in advance and subsequently estimating the pose. The majority of current deep learning-based image retrieval methods require labeled data, which increase data annotation costs and complicate the acquisition of data. In this paper, we propose an unsupervised hierarchical indoor localization framework that integrates an unsupervised network variational autoencoder (VAE) with a visual-based Structure-from-Motion (SfM) approach in order to extract global and local features. During the localization process, global features are applied for the image retrieval at the level of the scene map in order to obtain candidate images, and are subsequently used to estimate the pose from 2D-3D matches between query and candidate images. RGB images only are used as the input of the proposed localization system, which is both convenient and challenging. Experimental results reveal that the proposed method can localize images within 0.16 m and 4° in the 7-Scenes data sets and 32.8% within 5 m and 20° in the Baidu data set. Furthermore, our proposed method achieves a higher precision compared to advanced methods.

show abstract

“…A fine trained model can encode useful information from images into latent variable

Section: Methodsmentioning

confidence: 99%

“…Unlike autoencoders (AEs), VAE does not encode the training data as an isolated vector, rather it can force the latent variable to fill the space [ 45 ]. Therefore, the input images can be encoded in latent variables via the encoder network, which is useful for image retrieval [ 46 ] and clustering [ 47 , 48 ] tasks.…”

Section: Related Workmentioning

confidence: 99%

A Visual and VAE Based Hierarchical Indoor Localization Method

Jiang

Zou

Chen

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…It is noteworthy that the feature decomposing network has the latent space as its bottleneck, where no bypass connection between the encoder and decoders, such as skip connections (Drozdzal et al, 2016), is implemented. Therefore, we can expect that the information processed by the encoder can be compressed in latent spaces (Razavi et al, 2019;Wu and Flierl, 2019).…”

Section: Feature Decomposing Networkmentioning

confidence: 99%

Decomposing Normal and Abnormal Features of Medical Images into Discrete Latent Codes for Content-Based Image Retrieval

Kobayashi¹,

Hataya²,

Kurose³

et al. 2021

Preprint

View full text Add to dashboard Cite

In medical imaging, the characteristics purely derived from a disease should reflect the extent to which abnormal findings deviate from the normal features. Indeed, physicians often need corresponding images without abnormal findings of interest or, conversely, images that contain similar abnormal findings regardless of normal anatomical context. This is called comparative diagnostic reading of medical images, which is essential for a correct diagnosis. To support comparative diagnostic reading, content-based image retrieval (CBIR), which can selectively utilize normal and abnormal features in medical images as two separable semantic components, will be useful. Therefore, we propose a neural network architecture to decompose the semantic components of medical images into two latent codes: normal anatomy code and abnormal anatomy code. The normal anatomy code represents normal anatomies that should have existed if the sample is healthy, whereas the abnormal anatomy code attributes to abnormal changes that reflect deviation from the normal baseline. These latent codes are discretized through vector quantization to enable binary hashing, which can reduce the computational burden at the time of similarity search. By calculating the similarity based on either normal or abnormal anatomy codes or the combination of the two codes, our algorithm can retrieve images according to the selected semantic component from a dataset consisting of brain magnetic resonance images of gliomas. Our CBIR system qualitatively and quantitatively achieves remarkable results.

show abstract

“…The three techniques are not introduced by us and are already used by some existing work related to VQ but outside of the graph learning community. For example, product VQ is used in [45], and whitening is used in [46].…”

Section: Checklistmentioning

confidence: 99%

VQ-GNN: A Universal Framework to Scale up Graph Neural Networks using Vector Quantization

Ding¹,

Kong²,

Li³

et al. 2021

Preprint

View full text Add to dashboard Cite

Most state-of-the-art Graph Neural Networks (GNNs) can be defined as a form of graph convolution which can be realized by message passing between direct neighbors or beyond. To scale such GNNs to large graphs, various neighbor-, layer-, or subgraph-sampling techniques are proposed to alleviate the "neighbor explosion" problem by considering only a small subset of messages passed to the nodes in a mini-batch. However, sampling-based methods are difficult to apply to GNNs that utilize many-hops-away or global context each layer, show unstable performance for different tasks and datasets, and do not speed up model inference. We propose a principled and fundamentally different approach, VQ-GNN, a universal framework to scale up any convolution-based GNNs using Vector Quantization (VQ) without compromising the performance. In contrast to sampling-based techniques, our approach can effectively preserve all the messages passed to a mini-batch of nodes by learning and updating a small number of quantized reference vectors of global node representations, using VQ within each GNN layer. Our framework avoids the "neighbor explosion" problem of GNNs using quantized representations combined with a low-rank version of the graph convolution matrix. We show that such a compact low-rank version of the gigantic convolution matrix is sufficient both theoretically and experimentally. In company with VQ, we design a novel approximated message passing algorithm and a nontrivial back-propagation rule for our framework. Experiments on various types of GNN backbones demonstrate the scalability and competitive performance of our framework on large-graph node classification and link prediction benchmarks.

show abstract

Learning Product Codebooks Using Vector-Quantized Autoencoders for Image Retrieval

Cited by 14 publications

References 12 publications

A Visual and VAE Based Hierarchical Indoor Localization Method

A Visual and VAE Based Hierarchical Indoor Localization Method

Decomposing Normal and Abnormal Features of Medical Images into Discrete Latent Codes for Content-Based Image Retrieval

VQ-GNN: A Universal Framework to Scale up Graph Neural Networks using Vector Quantization

Contact Info

Product

Resources

About