Learning contextualized semantics from co-occurring terms via a Siamese architecture

Sandouk, Ubai; Chen, Ke

doi:10.1016/j.neunet.2016.01.004

Cited by 5 publications

(6 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zhang et al [30] put forward Fast0Tag model, to make the word vectors of relevant labels for a given image rank ahead of the irrelevant labels, along a principal direction in the word vector space. Sandouk et al [46] constructed a semantic learning model that capable of embedding an target label by inferring its meaning from its co-occurring labels. Lee et al [31] proposed a framework that incorporates knowledge graphs for describing the relationships between multiple labels.…”

Section: B Inductive Zero-shot Learningmentioning

confidence: 99%

Inductive Zero-Shot Image Annotation via Embedding Graph

et al. 2019

View full text Add to dashboard Cite

Conventional image annotation systems can only handle those images having labels within the exist library, but cannot recognize those novel labels. In order to learn new concepts, one has to gather large amount of labeled images and train the model from scratch. More importantly, it can come with a high price to collect those labeled images. For these reasons, we put forward a zero-shot image annotation model, to reduce the demand for the images with novel labels. In this paper, we focus on the two big challenges of zero-shot image annotation: polysemous words and a strong bias in the generalized zero-shot setting. For the first problem, instead of training on large corpus datasets as previous methods, we propose to adopt Node2Vec to obtain contextualized word embeddings, which can easily produce word vectors of the polysemous words. For the second problem, we alleviate the strong bias in two ways: on one hand, we utilize a model based on graph convolutional network (GCN) to make target images involved in the training process; on the other hand, we put forward a novel semantic coherent (SC) loss to capture the semantic relations of the source and target labels. The extensive experiments on NUSWIDE, COCO, IAPR TC-12, and Corel5k datasets show the superiority of the proposed model and the annotation performance get improved by 4%-6% comparing with state-of-the-art methods.

show abstract

Section: B Inductive Zero-shot Learningmentioning

confidence: 99%

Inductive Zero-Shot Image Annotation via Embedding Graph

et al. 2019

View full text Add to dashboard Cite

show abstract

“…For self-containment, we review our recently proposed approach for learning contextualized semantics from tags [Sandouk and Chen 2016]. As the underpinning techniques, this model is applied to learning semantics from musical tags.…”

Section: Model Descriptionmentioning

confidence: 99%

“…stops improving. See the appendix of [Sandouk and Chen 2016] for further details on learning algorithm.…”

Section: Siamese Architecture 22mentioning

confidence: 99%

“…In this paper, we investigate the suitability of this approach in learning musical semantics based on music tag collections. The main contributions in this work are summarized: -We thoroughly investigate the suitability of Siamese CE approach [Sandouk and Chen 2016] in modeling the musical concepts, including the proper capacity of the network in learning musical semantics, highlighting the emergent structure of the musical tag space and its smooth nature, and assessing the computational efficiency of the network in music domain. -To make a state-of-the-art contextualized semantic learning model comparable to ours, we propose an improved version for the CRBM model [Mandel et al 2011] so that it can capture statistical co-occurrence likelihoods of tags as our Siamese architecture does.…”

Section: Introductionmentioning

confidence: 99%

“…As a result, the specific meaning of a tag, dubbed a concept, is only identified after examining the tag and its local context. In our recent work, we formulate the concept embedding, (߬ , ߜ) , problem such that the distance between different concepts embedding reflects contextualized relatedness [Sandouk and Chen 2016]. To solve this problem, we proposed a contextualized semantic learning approach by means of Siamese architecture [Bromley et al 1993].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Learning Contextualized Music Semantics from Tags Via a Siamese Neural Network

Sandouk

Chen

2016

ACM Trans. Intell. Syst. Technol.

Self Cite

View full text Add to dashboard Cite

Music information retrieval faces a challenge in modeling contextualized musical concepts formulated by a set of co-occurring tags. In this paper, we investigate the suitability of our recently proposed approach based on a Siamese neural network in fighting off this challenge. By means of tag features and probabilistic topic models, the network captures contextualized semantics from tags via unsupervised learning. This leads to a distributed semantics space and a potential solution to the out of vocabulary problem which has yet to be sufficiently addressed. We explore the nature of the resultant music-based semantics and address computational needs. We conduct experiments on three public music tag collections -namely, CAL500, MagTag5K and Million Song Dataset-and compare our approach to a number of stateof-the-art semantics learning approaches. Comparative results suggest that this approach outperforms previous approaches in terms of semantic priming and music tag completion.

show abstract