Can Transformer be Too Compositional? Analysing Idiom Processing in Neural Machine Translation

Dankers, Verna; Lucas, Christopher G.; Titov, Ivan

doi:10.48550/arxiv.2205.15301

Cited by 2 publications

(2 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…But as shown in Table 3 we observe that BTG-Seq2Seq can translate proper nouns, time expression, and even idioms to an extent. The idiom case is particularly interesting as pure seq2seq systems are known to be too compositional when translating idioms (Dankers et al, 2022).…”

Section: Discussionmentioning

confidence: 99%

Hierarchical Phrase-based Sequence-to-Sequence Learning

Wang¹,

Titov²,

Andreas³

et al. 2022

Preprint

View full text Add to dashboard Cite

We describe a neural transducer that maintains the flexibility of standard sequence-tosequence (seq2seq) models while incorporating hierarchical phrases as a source of inductive bias during training and as explicit constraints during inference. Our approach trains two models: a discriminative parser based on a bracketing transduction grammar whose derivation tree hierarchically aligns source and target phrases, and a neural seq2seq model that learns to translate the aligned phrases one-byone. We use the same seq2seq model to translate at all phrase scales, which results in two inference modes: one mode in which the parser is discarded and only the seq2seq component is used at the sequence-level, and another in which the parser is combined with the seq2seq model. Decoding in the latter mode is done with the cube-pruned CKY algorithm, which is more involved but can make use of new translation rules during inference. We formalize our model as a source-conditioned synchronous grammar and develop an efficient variational inference algorithm for training. When applied on top of both randomly initialized and pretrained seq2seq models, we find that both inference modes performs well compared to baselines on small scale machine translation benchmarks.

show abstract

Section: Discussionmentioning

confidence: 99%

Hierarchical Phrase-based Sequence-to-Sequence Learning

Wang¹,

Titov²,

Andreas³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Given that figurativeness is commonplace in everyday communication (Lakoff and Johnson, 2008), progress in the field of Natural Language Understanding (NLU) would be incomplete without figurativeness understanding. Consequently, figurative text has been studied in various downstream NLP tasks such as machine translation (Dankers et al, 2022), textual entailment (Agerri, 2008), (Chakrabarty et al, 2021), (Liu et al, 2022) and dialog models (Jhamtani et al, 2021), inter-alia. However, to the best of our knowledge, there has not been a systematic study of figurative language understanding capabilities of question answering models.…”

Section: -Oscar Wildementioning

confidence: 99%

Does the “most sinfully decadent cake ever” taste good? Answering Yes/No Questions from Figurative Contexts

Rakshit,

Flanigan

2023

Proceedings of the Conference Recent Advances in Natural Language Processing - Large Language Models for Natural Language Proce

View full text Add to dashboard Cite

Figurative language is commonplace in natural language, and while making communication memorable and creative, can be difficult to understand. In this work, we investigate the robustness of Question Answering (QA) models on figurative text. Yes/no questions, in particular, are a useful probe of figurative language understanding capabilities of large language models. We propose FigurativeQA, a set of 1000 yes/no questions with figurative and nonfigurative contexts, extracted from the domains of restaurant and product reviews. We show that state-of-the-art BERT-based QA models exhibit an average performance drop of up to 15% points when answering questions from figurative contexts, as compared to non-figurative ones. While models like GPT-3 and ChatGPT are better at handling figurative texts, we show that further performance gains can be achieved by automatically simplifying the figurative contexts into their non-figurative (literal) counterparts. We find that the best overall model is ChatGPT with chain-of-thought prompting to generate non-figurative contexts. Our work provides a promising direction for building more robust QA models with figurative language understanding capabilities.

show abstract

Can Transformer be Too Compositional? Analysing Idiom Processing in Neural Machine Translation

Cited by 2 publications

References 30 publications

Hierarchical Phrase-based Sequence-to-Sequence Learning

Hierarchical Phrase-based Sequence-to-Sequence Learning

Does the “most sinfully decadent cake ever” taste good? Answering Yes/No Questions from Figurative Contexts

Contact Info

Product

Resources

About