FLUTE: Figurative Language Understanding through Textual Explanations

Chakrabarty, Tuhin; Saakyan, Arkadiy; Ghosh, Debanjan; Muresan, Smaranda

doi:10.18653/v1/2022.emnlp-main.481

Cited by 10 publications

(4 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With the script representation, we describe how to embed a cultural norm into a sentence context to generate a premise and hypothesis pair. The structured representation of cultural norm allows us to leverage LLMs to generate premises and hypotheses, a promising approach to construct NLI datasets (Liu et al, 2022;Chakrabarty et al, 2022). Specifically, we first prompt ChatGPT/GPT-3.5turbo to generate premises and hypotheses from the script, followed by human editing on the generated content.…”

Section: Model-in-the-loop Generationmentioning

confidence: 99%

Culturally Aware Natural Language Inference

Huang,

Yang

2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

Humans produce and consume language in a particular cultural context, which includes knowledge about specific norms and practices. A listener's awareness of the cultural context is critical for interpreting the speaker's meaning. A simple expression like "I didn't leave a tip" implies a strong sense of dissatisfaction when tipping is assumed to be the norm. As NLP systems reach users from different cultures, achieving culturally aware language understanding becomes increasingly important. However, current research has focused on building cultural knowledge bases without studying how such knowledge leads to contextualized interpretations of texts. In this work, we operationalize cultural variations in language understanding through a natural language inference (NLI) task that surfaces cultural variations as label disagreement between annotators from different cultural groups. We introduce the first Culturally Aware Natural Language Inference (CALI) dataset with 2.7K premise-hypothesis pairs annotated by two cultural groups located in the U.S. and India. With CALI, we categorize how cultural norms affect language understanding and present an evaluation framework to assess at which levels large language models are culturally aware. Our dataset is available at https://github.com/ SALT-NLP/CulturallyAwareNLI.

show abstract

Section: Model-in-the-loop Generationmentioning

confidence: 99%

Culturally Aware Natural Language Inference

Huang,

Yang

2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

show abstract

“…Here, we structure the identification of social norm differences across cultures as an explainable textual entailment task (a.k.a natural language inference (NLI)) similar to e-SNLI (Camburu et al, 2018); a difference in cultural norms equates to a contradiction between norms of different cultures for a given situation, and vice versa for entailment. Given the recent success of human-AI collaboration frameworks (Wiegreffe et al, 2022;Bartolo et al, 2022;Chakrabarty et al, 2022), the complex nature of Given an input Text, respond with Social Norms that are assumed by the speaker of the text, in English. Social Norms are rules and standards that are understood by members of a group, and that guide or constrain social behaviors without the force of law.…”

Section: Data Sourcesmentioning

confidence: 99%

“…Prior work in interpretability (Wiegreffe et al, 2021) has shown that rationales from joint selfrationalizing models-predicting both the explanation alongside the relation label-are capable of producing faithful free-text rationales. Following prior work (Chakrabarty et al, 2022), we fine-tune a joint self-rationalizing T5 model in multiple settings, randomly splitting our data into 65% train, 10% validation, and 25% test set splits (Appendix Section C).…”

Section: Experiments and Evaluationmentioning

confidence: 99%

Sociocultural Norm Similarities and Differences via Situational Alignment and Explainable Textual Entailment

CH-Wang,

Saakyan,

et al. 2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Designing systems that can reason across cultures requires that they are grounded in the norms of the contexts in which they operate. However, current research on developing computational models of social norms has primarily focused on American society. Here, we propose a novel approach to discover and compare descriptive social norms across Chinese and American cultures. We demonstrate our approach by leveraging discussions on a Chinese Q&A platform-知乎 (Zhihu)-and the existing SOCIALCHEMISTRY dataset as proxies for contrasting cultural axes, align social situations cross-culturally, and extract social norms from texts using in-context learning. Embedding Chain-of-Thought prompting in a human-AI collaborative framework, we build a high-quality dataset of 3,069 social norms aligned with social situations across Chinese and American cultures alongside corresponding free-text explanations. To test the ability of models to reason about social norms across cultures, we introduce the task of explainable social norm entailment, showing that existing models under 3B parameters have significant room for improvement in both automatic and human evaluation. Further analysis of crosscultural norm differences based on our dataset shows empirical alignment with the social orientations framework, revealing several situational and descriptive nuances in norms across these cultures.

show abstract

“…Numerous sources have been explored to capture figurative expressions (Chakrabarty et al, 2022;Liu et al, 2022b;Bizzoni and Lappin, 2018). Nonetheless, they often suffer from limitations in scale or cost.…”

Section: Data Collectionmentioning

confidence: 99%

ViPE: Visualise Pretty-much Everything

Shahmohammadi,

Ghosh,

Lensch

2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Figurative and non-literal expressions are profoundly integrated in human communication.Visualising such expressions allow us to convey our creative thoughts, and evoke nuanced emotions. Recent text-to-image models like Stable Diffusion, on the other hand, struggle to depict non-literal expressions. Recent works primarily deal with this issue by compiling humanly annotated datasets on a small scale, which not only demands specialised expertise but also proves highly inefficient. To address this issue, we introduce ViPE: Visualise Pretty-much Everything. ViPE offers a series of lightweight and robust language models that have been trained on a large-scale set of lyrics with noisy visual descriptions that represent their implicit meaning. The synthetic visual descriptions are generated by GPT3.5 relying on neither human annotations nor images. ViPE effectively expresses any arbitrary piece of text into a visualisable description, enabling meaningful and high-quality image generation. We provide compelling evidence that ViPE is more robust than GPT3.5 in synthesising visual elaborations. ViPE also exhibits an understanding of figurative expressions comparable to human experts, providing a powerful and open-source backbone to many downstream applications such as music video and caption generation.

show abstract

FLUTE: Figurative Language Understanding through Textual Explanations

Cited by 10 publications

References 31 publications

Culturally Aware Natural Language Inference

Culturally Aware Natural Language Inference

Sociocultural Norm Similarities and Differences via Situational Alignment and Explainable Textual Entailment

ViPE: Visualise Pretty-much Everything

Contact Info

Product

Resources

About