Graph Transformer for Graph-to-Sequence Learning

Cai, Deng; Lam, Wai

doi:10.1609/aaai.v34i05.6243

Cited by 170 publications

(164 citation statements)

References 20 publications

Supporting

Mentioning

163

Contrasting

Unclassified

Order By: Relevance

“…configurations of our model are superior to the baselines by a significant margin. Noticeably, DCGCN and graph transformer are strong baselines, delivering SOTA performance across tasks such as AMRto-text generation and syntax-based neural machine translation (Guo et al, 2019;Cai and Lam, 2019). We believe the larger number of edge types in our task impairs their capability.…”

Section: Methodsmentioning

confidence: 98%

“…In the "Modified GraphRNN" baseline (iii), we use the breadth-first-search (BFS) based node ordering to flatten the graph 4 , and use RNNs as the encoders (You et al, 2018) and a decoder similar to our systems. In the final two baselines, "Graph Transformer" (iv) and "Deep Convolutional Graph Networks" (DCGCN) (v), we use the Graph Transformers (Cai and Lam, 2019) and Deep Convolutional Graph Networks (Guo et al, 2019) to encode the source graph (the decoder is identical to ours).…”

Section: Methodsmentioning

confidence: 99%

“…Our work focuses on the update of scene graphs based on users' queries, while previous works model the modifications of semantic representations in multi-turn dialogue systems. Due to their effectiveness, GCNs and graph transformer have been used as graph encoder for graphto-sequence transduction in semantic-based text generation (Bastings et al, 2017;Beck et al, 2018;Guo et al, 2019;Cai and Lam, 2019;Song et al, 2018;Wu et al, 2020).…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Scene Graph Modification Based on Natural Language Commands

Tran

Haffari

et al. 2020

Findings of the Association for Computational Linguistics: EMNLP 2020

View full text Add to dashboard Cite

Structured representations like graphs and parse trees play a crucial role in many Natural Language Processing systems. In recent years, the advancements in multi-turn user interfaces necessitate the need for controlling and updating these structured representations given new sources of information. Although there have been many efforts focusing on improving the performance of the parsers that map text to graphs or parse trees, very few have explored the problem of directly manipulating these representations. In this paper, we explore the novel problem of graph modification, where the systems need to learn how to update an existing scene graph given a new user's command. Our novel models based on graph-based sparse transformer and cross attention information fusion outperform previous systems adapted from the machine translation and graph generation literature. We further contribute our large graph modification datasets to the research community to encourage future research for this new problem.

show abstract

Section: Methodsmentioning

confidence: 98%

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Scene Graph Modification Based on Natural Language Commands

Tran

Haffari

et al. 2020

Findings of the Association for Computational Linguistics: EMNLP 2020

View full text Add to dashboard Cite

show abstract

“…We take a graph Transformer model (Koncel-Kedziorski et al, 2019;Zhu et al, 2019;Cai and Lam, 2020;Wang et al, 2020) as our baseline. Previous work has proposed several variations of graph-Transformer.…”

Section: Baseline: Graph Transformermentioning

confidence: 99%

“…In particular, graph neural networks (Beck et al, 2018;Song et al, 2018;Guo et al, 2019) and richer graph representations (Damonte and Cohen, 2019;Hajdik et al, 2019;Ribeiro et al, 2019) have been shown to give better performances than RNN-based models (Konstas et al, 2017) on linearized graphs. Subsequent work exploited graph Transformer (Zhu et al, 2019;Cai and Lam, 2020;Wang et al, 2020), achieving better performances by directly modeling the intercorrelations between distant node pairs with relation-aware global communication. Despite the progress on the encoder side, the current stateof-the-art models use a rather standard decoder: it functions as a language model, where each word is generated given only the previous words.…”

Section: Introductionmentioning

confidence: 99%

Online Back-Parsing for AMR-to-Text Generation

Bai

Song

Zhang

2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

AMR-to-text generation aims to recover a text containing the same meaning as an input AMR graph. Current research develops increasingly powerful graph encoders to better represent AMR graphs, with decoders based on standard language modeling being used to generate outputs. We propose a decoder that back predicts projected AMR graphs on the target sentence during text generation. As the result, our outputs can better preserve the input meaning than standard decoders. Experiments on two AMR benchmarks show the superiority of our model over the previous state-of-the-art system based on graph Transformer.

show abstract

Transformer technology in molecular science

Jiang,

Ke,

Chen

et al. 2024

WIREs Comput Mol Sci

View full text Add to dashboard Cite

A transformer is the foundational architecture behind large language models designed to handle sequential data by using mechanisms of self‐attention to weigh the importance of different elements, enabling efficient processing and understanding of complex patterns. Recently, transformer‐based models have become some of the most popular and powerful deep learning (DL) algorithms in molecular science, owing to their distinctive architectural characteristics and proficiency in handling intricate data. These models leverage the capacity of transformer architectures to capture complex hierarchical dependencies within sequential data. As the applications of transformers in molecular science are very widespread, in this review, we only focus on the technical aspects of transformer technology in molecule domain. Specifically, we will provide an in‐depth investigation into the algorithms of transformer‐based machine learning techniques in molecular science. The models under consideration include generative pre‐trained transformer (GPT), bidirectional and auto‐regressive transformers (BART), bidirectional encoder representations from transformers (BERT), graph transformer, transformer‐XL, text‐to‐text transfer transformer, vision transformers (ViT), detection transformer (DETR), conformer, contrastive language‐image pre‐training (CLIP), sparse transformers, and mobile and efficient transformers. By examining the inner workings of these models, we aim to elucidate how their architectural innovations contribute to their effectiveness in processing complex molecular data. We will also discuss promising trends in transformer models within the context of molecular science, emphasizing their technical capabilities and potential for interdisciplinary research. This review seeks to provide a comprehensive understanding of the transformer‐based machine learning techniques that are driving advancements in molecular science.This article is categorized under: Data Science > Chemoinformatics Data Science > Artificial Intelligence/Machine Learning

show abstract

Graph Transformer for Graph-to-Sequence Learning

Cited by 170 publications

References 20 publications

Scene Graph Modification Based on Natural Language Commands

Scene Graph Modification Based on Natural Language Commands

Online Back-Parsing for AMR-to-Text Generation

Transformer technology in molecular science

Contact Info

Product

Resources

About