Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks

Tai, Kai Sheng; Socher, Richard; Manning, Christopher D.

doi:10.48550/arxiv.1503.00075

Cited by 247 publications

(374 citation statements)

References 0 publications

Supporting

Mentioning

372

Contrasting

Unclassified

Order By: Relevance

“…CNN-based models focus on patterns of words across space, whereas RNN-based ones can better capture time-level features of words. Tai et al [13] introduced a generalization of the standard Long Short-Term Memory (LSTM) architecture to tree-structured network topologies, which achieves high performance for representing sentence meaning over a sequential LSTM.…”

Section: Text Representation Learningmentioning

confidence: 99%

SAM: A Self-adaptive Attention Module for Context-Aware Recommendation System

Liu,

Wei,

et al. 2021

Preprint

View full text Add to dashboard Cite

Recently, textual information has been proved to play a positive role in recommendation systems. However, most of the existing methods only focus on representation learning of textual information in ratings, while potential selection bias induced by the textual information is ignored. In this work, we propose a novel and general self-adaptive module, the Self-adaptive Attention Module (SAM), which adjusts the selection bias by capturing contextual information based on its representation. This module can be embedded into recommendation systems that contain learning components of contextual information. Experimental results on three real-world datasets demonstrate the effectiveness of our proposal, and the state-of-the-art models with SAM significantly outperform the original ones. * Jiabin Liu and Zheng Wei contributed equally to this work. † Xiaojun Mao and Jian Wang are the co-corresponding authors.

show abstract

Section: Text Representation Learningmentioning

confidence: 99%

SAM: A Self-adaptive Attention Module for Context-Aware Recommendation System

Liu,

Wei,

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Since an AST could fully represent the lexical and grammatical structure of the source code without recording all details of the code (such as punctuation marks), it is widely used in many software engineering techniques/tools. Some AST-based deep learning code embedding techniques include RvNN, TBCNN, Tree-LSTM, ASTNN and code2vec [15], [16], [18]- [20]. In this paper, we mainly use code2vec to represent the semantics of method code.…”

Section: B Ast-based Code Embeddingmentioning

confidence: 99%

BLESER: Bug Localization Based on Enhanced Semantic Retrieval

Zou,

Li,

Fang

2021

Preprint

View full text Add to dashboard Cite

“…A next generation of methods use word embedding techniques, like the word2vec (Mikolov et al, 2013a,b) and GloVe (Pennington et al, 2014), which furthermore consider the neighborhood relation of words and can benefit from external corpora for training the representations. Another line of work focuses on the sequential structure of tokens within text and uses deep learning architectures like the CNN's, as in (Kim, 2014), or RNN's, as e.g., in (Tai et al, 2015) to capture semantic information.…”

Section: Related Work 21 Text Classificationmentioning

confidence: 99%

Classification of hierarchical text using geometric deep learning: the case of clinical trials corpus

Ferdowsi¹,

Borissov²,

Knafou³

et al. 2021

Preprint

View full text Add to dashboard Cite

We consider the hierarchical representation of documents as graphs and use geometric deep learning to classify them into different categories. While graph neural networks can efficiently handle the variable structure of hierarchical documents using the permutation invariant message passing operations, we show that we can gain extra performance improvements using our proposed selective graph pooling operation that arises from the fact that some parts of the hierarchy are invariable across different documents. We applied our model to classify clinical trial (CT) protocols into completed and terminated categories. We use bag-of-words based, as well as pre-trained transformer-based embeddings to featurize the graph nodes, achieving f1-scores 0.85 on a publicly available large scale CT registry of around 360K protocols. We further demonstrate how the selective pooling can add insights into the CT termination status prediction. We make the source code and dataset splits accessible.

show abstract

Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks

Cited by 247 publications

References 0 publications

SAM: A Self-adaptive Attention Module for Context-Aware Recommendation System

SAM: A Self-adaptive Attention Module for Context-Aware Recommendation System

BLESER: Bug Localization Based on Enhanced Semantic Retrieval

Classification of hierarchical text using geometric deep learning: the case of clinical trials corpus

Contact Info

Product

Resources

About