NASH: Toward End-to-End Neural Architecture for Generative Semantic Hashing

Shen, Dinghan; Su, Qinliang; Chapfuwa, Paidamoyo; Wang, Wenlin; Wang, Guoyin; Carin, Lawrence; Henao, Ricardo

doi:10.48550/arxiv.1805.05361

Cited by 5 publications

(9 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…But due to the continuous characteristics of Gaussian random variables, a separate binarization step is required to transform the continuous latent representations into binary codes. To overcome the separate training issue, Bernoulli prior and posterior are then proposed in NASH (Shen et al, 2018). With the recent advances on gradient estimators for discrete random variables, the model successfully circumvents gradient backpropagation issue for discrete variables, and can be trained efficiently in an end-to-end manner.…”

Section: Preliminaries On Generative Hashing For Documentsmentioning

confidence: 99%

“…However, the two-stage training procedure is prone to undermine the performance. NASH (Shen et al, 2018) tackled this issue by replacing Gaussian prior with Bernoulli in VAE and adopting straight-through to enable end-to-end training. Since then, a lot of methods surged to improve the performance.…”

Section: Related Workmentioning

confidence: 99%

“…Baselines We compare our model with the following unsupervised deep semantic hashing methods: VDSH (Chaidaroon and Fang, 2017), NASH (Shen et al, 2018), BMSH (Dong et al, 2019), WISH (Ye et al, 2020), CorrSH (Zheng et al, 2020) and AMMI (Stratos and Wiseman, 2020).…”

Section: Experiments Setupmentioning

confidence: 99%

“…Currently, most of unsupervised document hashing methods are established upon the perspective of deep generative models (Kingma and Welling, 2013;Rezende et al, 2014). Essentially, all these methods seek to model the documents with a deep generative model and then employ the latent representations of documents to construct hash codes (Chaidaroon and Fang, 2017;Shen et al, 2018;Dong et al, 2019;Ye et al, 2020;Zheng et al, 2020;Ou et al, 2021). Although great successes have been observed in these methods, due to the difficulties in capturing the long dependency structures of words (especially for long documents), all of these methods are established on modeling the BOW or TFIDF features of documents.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Refining BERT Embeddings for Document Hashing via Mutual Information Maximization

Ou¹,

Su²,

Yu³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Existing unsupervised document hashing methods are mostly established on generative models. Due to the difficulties of capturing long dependency structures, these methods rarely model the raw documents directly, but instead to model the features extracted from them (e.g. bag-of-words (BOW), TFIDF). In this paper, we propose to learn hash codes from BERT embeddings after observing their tremendous successes on downstream tasks. As a first try, we modify existing generative hashing models to accommodate the BERT embeddings. However, little improvement is observed over the codes learned from the old BOW or TFIDF features. We attribute this to the reconstruction requirement in the generative hashing, which will enforce irrelevant information that is abundant in the BERT embeddings also compressed into the codes. To remedy this issue, a new unsupervised hashing paradigm is further proposed based on the mutual information (MI) maximization principle. Specifically, the method first constructs appropriate global and local codes from the documents and then seeks to maximize their mutual information. Experimental results on three benchmark datasets demonstrate that the proposed method is able to generate hash codes that outperform existing ones learned from BOW features by a substantial margin. 1

show abstract

Section: Preliminaries On Generative Hashing For Documentsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Experiments Setupmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Refining BERT Embeddings for Document Hashing via Mutual Information Maximization

Ou¹,

Su²,

Yu³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Variational Autoencoder VAE [21] is a powerful framework for learning stochastic representations that account for model uncertainty. While its applications have been extensively studied in the context of computer vision and NLP [33,34,45,51], its use in complex network analysis is less widely explored. Existing solutions focused on building VAEs for the generation of a graph, but not the associated contents [22,26,36].…”

Section: Related Workmentioning

confidence: 99%

Improving Textual Network Learning with Variational Homophilic Embeddings

Wang,

Tao,

Gan

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

The performance of many network learning applications crucially hinges on the success of network embedding algorithms, which aim to encode rich network information into low-dimensional vertex-based vector representations. This paper considers a novel variational formulation of network embeddings, with special focus on textual networks. Different from most existing methods that optimize a discriminative objective, we introduce Variational Homophilic Embedding (VHE), a fully generative model that learns network embeddings by modeling the semantic (textual) information with a variational autoencoder, while accounting for the structural (topology) information through a novel homophilic prior design. Homophilic vertex embeddings encourage similar embedding vectors for related (connected) vertices. The proposed VHE promises better generalization for downstream tasks, robustness to incomplete observations, and the ability to generalize to unseen vertices. Extensive experiments on real-world networks, for multiple tasks, demonstrate that the proposed method consistently achieves superior performance relative to competing state-of-the-art approaches.

show abstract

Learning to Hash with Graph Neural Networks for Recommender Systems

Tan

Liu

Zhao

et al. 2020

Proceedings of the Web Conference 2020

View full text Add to dashboard Cite

Recommender systems in industry generally include two stages: recall and ranking. Recall refers to efficiently identify hundreds of candidate items that user may interest in from a large volume of item corpus, while the latter aims to output a precise ranking list using complex ranking models. Recently, graph representation learning has attracted much attention in supporting high quality candidate search at scale. Despite its effectiveness in learning embedding vectors for objects in the user-item interaction network, the computational costs to infer users' preferences in continuous embedding space are tremendous. In this work, we investigate the problem of hashing with graph neural networks (GNNs) for high quality retrieval, and propose a simple yet effective discrete representation learning framework to jointly learn continuous and discrete codes. Specifically, a deep hashing with GNNs (HashGNN) is presented, which consists of two components, a GNN encoder for learning node representations, and a hash layer for encoding representations to hash codes. The whole architecture is trained end-to-end by jointly optimizing two losses, i.e., reconstruction loss from reconstructing observed links, and ranking loss from preserving the relative ordering of hash codes. A novel discrete optimization strategy based on straight through estimator (STE) with guidance is proposed. The principal idea is to avoid gradient magnification in back-propagation of STE with continuous embedding guidance, in which we begin from learning an easier network that mimic the continuous embedding and let it evolve during the training until it finally goes back to STE. Comprehensive experiments over three publicly available and one real-world Alibaba company datasets demonstrate that our model not only can achieve comparable performance compared with its continuous counterpart but also runs multiple times faster during inference.

show abstract

NASH: Toward End-to-End Neural Architecture for Generative Semantic Hashing

Cited by 5 publications

References 17 publications

Refining BERT Embeddings for Document Hashing via Mutual Information Maximization

Refining BERT Embeddings for Document Hashing via Mutual Information Maximization

Improving Textual Network Learning with Variational Homophilic Embeddings

Learning to Hash with Graph Neural Networks for Recommender Systems

Contact Info

Product

Resources

About