Learning to Embed Sentences Using Attentive Recursive Trees

Shi, Jiaxin; Hou, Lei; Li, Juanzi; Liu, Zhiyuan; Zhang, Hanwang

doi:10.1609/aaai.v33i01.33016991

Cited by 3 publications

(2 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Linguistic structure induction from text. Recent work has proposed several approaches for inducing latent syntactic structures, including constituency trees (Choi et al, 2018;Yogatama et al, 2017;Maillard and Clark, 2018;Havrylov et al, 2019;Kim et al, 2019;Drozdov et al, 2019) and dependency trees (Shi et al, 2019), from the distant supervision of downstream tasks. However, most of the methods are not able to produce linguistically sound structures, or even consistent ones with fixed data and hyperparameters but different random initializations (Williams et al, 2018).…”

Section: Related Workmentioning

confidence: 99%

Visually Grounded Neural Syntax Acquisition

Shi¹,

Mao²,

Gimpel³

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

We present the Visually Grounded Neural Syntax Learner (VG-NSL), an approach for learning syntactic representations and structures without explicit supervision. The model learns by looking at natural images and reading paired captions. VG-NSL generates constituency parse trees of texts, recursively composes representations for constituents, and matches them with images. We define the concreteness of constituents by their matching scores with images, and use it to guide the parsing of text. Experiments on the MSCOCO data set show that VG-NSL outperforms various unsupervised parsing approaches that do not use visual grounding, in terms of F 1 scores against gold parse trees. We find that VG-NSL is much more stable with respect to the choice of random initialization and the amount of training data. We also find that the concreteness acquired by VG-NSL correlates well with a similar measure defined by linguists. Finally, we also apply VG-NSL to multiple languages in the Multi30K data set, showing that our model consistently outperforms prior unsupervised approaches.

show abstract

Section: Related Workmentioning

confidence: 99%

Visually Grounded Neural Syntax Acquisition

Shi¹,

Mao²,

Gimpel³

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

show abstract

“…Liu et al [16] proposed attentive tree-structured LSTM for VQA. To address the unbalanced distribution of weights, Shi et al [17] proposed an attentive recursive neural network for sentence embedding, which integrated task-specific attention mechanism into Tree-LSTM. Geng et al [18] utilized attentive Tree-LSTM and sequential respectively to extract semantic relation, and proved the effectiveness of the attention mechanism.…”

Section: Related Workmentioning

confidence: 99%

Child_Sum EATree-LSTMs: Enhanced Attentive Child_Sum Tree-LSTMs for Biomedical Event Extraction

Wang

Cao

Liu

et al. 2023

Preprint

View full text Add to dashboard Cite

Background The tree-structured neural network can deeply extract lexical representations of sentence syntactic structure. Some studies have utilized Recursive Neural Network to detect event triggers. Methods We incorporate the attention mechanism into Child-Sum Tree-LSTMs for the task of biomedical event triggers. Based on the previous research, we incorporated attention mechanism into Child-Sum Tree-LSTMs to assign an attention weight for the adjacent nodes to detect the biomedical event trigger words. The existing shallow syntactic dependencies in Child-Sum Tree-LSTMs ignore the deep syntactic dependencies. To enhance the effect of attention mechanism, we integrate the enhanced attention mechanism into the Child-Sum Tree-LSTMs model using the deep syntactic dependencies. Results Our proposed model integrating an enhanced the attention mechanism in Tree-LSTM on MLEE and BioNLP’09 both show best performance. The model also achieves the better performance on almost all of the complex event categories on the test set of BioNLP’09/11/13. Conclusion We evaluate the model performance on the MLEE and BioNLP datasets, and the experimental results demonstrate the advantage of enhanced attention to detect biomedical event trigger words.

show abstract

Child-Sum EATree-LSTMs: enhanced attentive Child-Sum Tree-LSTMs for biomedical event extraction

et al. 2023

View full text Add to dashboard Cite

Background Tree-structured neural networks have shown promise in extracting lexical representations of sentence syntactic structures, particularly in the detection of event triggers using recursive neural networks. Methods In this study, we introduce an attention mechanism into Child-Sum Tree-LSTMs for the detection of biomedical event triggers. We incorporate previous researches on assigning attention weights to adjacent nodes and integrate this mechanism into Child-Sum Tree-LSTMs to improve the detection of event trigger words. We also address a limitation of shallow syntactic dependencies in Child-Sum Tree-LSTMs by integrating deep syntactic dependencies to enhance the effect of the attention mechanism. Results Our proposed model, which integrates an enhanced attention mechanism into Tree-LSTM, shows the best performance for the MLEE and BioNLP’09 datasets. Moreover, our model outperforms almost all complex event categories for the BioNLP’09/11/13 test set. Conclusion We evaluate the performance of our proposed model with the MLEE and BioNLP datasets and demonstrate the advantage of an enhanced attention mechanism in detecting biomedical event trigger words.

show abstract

Learning to Embed Sentences Using Attentive Recursive Trees

Cited by 3 publications

References 21 publications

Visually Grounded Neural Syntax Acquisition

Visually Grounded Neural Syntax Acquisition

Child_Sum EATree-LSTMs: Enhanced Attentive Child_Sum Tree-LSTMs for Biomedical Event Extraction

Child-Sum EATree-LSTMs: enhanced attentive Child-Sum Tree-LSTMs for biomedical event extraction

Contact Info

Product

Resources

About