Assessing Multilingual Fairness in Pre-trained Multimodal Representations

Wang, Jialu; Liu, Yang; Wang, Xin Eric

doi:10.18653/v1/2022.findings-acl.211

Cited by 8 publications

(4 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zhao et al (2020) found that when a model trained on one language is deployed to another language, it also carries bais from the source to the target languages. Wang et al (2022) considered languages as fair objects and found that multimodal models are biased against different languages. Cho et al ( 2021) examined the issue of gender bias in translation systems in German, Korean, Portuguese, and Tagalog and found that scaling up language resources may amplify the bias cross-linguistically.…”

Section: Research Of Multilingual Text Debiasing Methodsmentioning

confidence: 99%

Model and Evaluation: Towards Fairness in Multilingual Text Classification

Lin¹,

He²,

Tang³

et al. 2023

Preprint

View full text Add to dashboard Cite

Recently, more and more research has focused on addressing bias in text classification models. However, existing research mainly focuses on the fairness of monolingual text classification models, and research on fairness for multilingual text classification is still very limited. In this paper, we focus on the task of multilingual text classification and propose a debiasing framework for multilingual text classification based on contrastive learning. Our proposed method does not rely on any external language resources and can be extended to any other languages. The model contains four modules: multilingual text representation module, language fusion module, text debiasing module, and text classification module. The multilingual text representation module uses a multilingual pre-trained language model to represent the text, the language fusion module makes the semantic spaces of different languages tend to be consistent through contrastive learning, and the text debiasing ©2000 Marina Meilȃ and Michael I. Jordan.

show abstract

Section: Research Of Multilingual Text Debiasing Methodsmentioning

confidence: 99%

Model and Evaluation: Towards Fairness in Multilingual Text Classification

Lin¹,

He²,

Tang³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…2) Approach#2: We adopted a methodology similar to Approach#1, with a modification involving the utilization of Arabic pre-trained encoders ( [24], [41]) and multilingual pre-trained language models [42] to extract the vector representations for each paragraph p j and question q. Notably, this approach involves obtaining these vectors without fine-tuning the model on the Arabic-NarrativeQA dataset.…”

Section: Arabic-narrativeqa Systemmentioning

confidence: 99%

Arabic Narrative Question Answering (QA) Using Transformer Models

Ateeq,

Tiun,

Abdelhaq

et al. 2024

IEEE Access

View full text Add to dashboard Cite

The Narrative question answering (QA) problem involves generating accurate, relevant, and human-like answers to questions based on the comprehension of a story consisting of logically connected paragraphs. Developing Narrative QA models allows students to ask about inconspicuous narrative elements while reading the story. However, this problem remains unexplored for the Arabic language because of the lack of Arabic narrative datasets. To address this gap, we present the Arabic-NarrativeQA dataset, which is the first dataset specifically designed for machine-reading comprehension of Arabic stories. This dataset consists of two parts: translation of an English NarrativeQA dataset and a collection of new question-answer pairs based on Arabic stories. Furthermore, we implement the Arabic-NarrativeQA system using the Ranker-Reader pipeline, exploring and evaluating various approaches at each stage to identify the most effective ones. To avoid the need for an extensive data collection process, we utilize cross-lingual transfer learning techniques to leverage knowledge transfer from the English Narrative QA dataset to the Arabic-NarrativeQA system. Experiments show that incorporating cross-lingual transfer learning significantly improved the performance of the reader models. Furthermore, the question's evidence information provided in the Arabic-NarrativeQA dataset enables the learnable rankers to effectively identify and select the pertinent paragraphs. Finally, we examine and categorize challenging questions that require a deep understanding of the stories. By incorporating these question types into the introduced dataset, we show that existing reading comprehension models struggle to answer them, and further model development should be conducted. To promote further research on this task, we make both the Arabic-NarrativeQA dataset and the pre-trained models publicly available.

show abstract

“…• We study the automatic evaluation based on a "general" vision-language model CLIP, which is flexible and powerful. Consequently, this also limits the upper bound performance of our metrics as CLIP is not trained for classifying attributes and might contain bias itself (Agarwal et al, 2021;Wang et al, 2021Yamada et al, 2022). For a more accurate evaluation, leveraging task-specific models (Chia et al, 2022) based on the fashion research in computer vision (Liu et al, 2016;Cheng et al, 2021) could be helpful, which we leave for future exploration.…”

Section: Limitationsmentioning

confidence: 99%

Auditing Gender Presentation Differences in Text-to-Image Models

Zhang¹,

J²,

Turk³

et al. 2023

Preprint

View full text Add to dashboard Cite

Text-to-image models, which can generate high-quality images based on textual input, have recently enabled various content-creation tools. Despite significantly affecting a wide range of downstream applications, the distributions of these generated images are still not fully understood, especially when it comes to the potential stereotypical attributes of different genders. In this work, we propose a paradigm (Gender Presentation Differences) that utilizes fine-grained self-presentation attributes to study how gender is presented differently in text-to-image models. By probing gender indicators in the input text (e.g., "a woman" or "a man"), we quantify the frequency differences of presentation-centric attributes (e.g., "a shirt" and "a dress") through human annotation and introduce a novel metric: GEP. 1 Furthermore, we propose an automatic method to estimate such differences. The automatic GEP metric based on our approach yields a higher correlation with human annotations than that based on existing CLIP scores, consistently across three state-of-theart text-to-image models. Finally, we demonstrate the generalization ability of our metrics in the context of gender stereotypes related to occupations. 2

show abstract

Assessing Multilingual Fairness in Pre-trained Multimodal Representations

Cited by 8 publications

References 29 publications

Model and Evaluation: Towards Fairness in Multilingual Text Classification

Model and Evaluation: Towards Fairness in Multilingual Text Classification

Arabic Narrative Question Answering (QA) Using Transformer Models

Auditing Gender Presentation Differences in Text-to-Image Models

Contact Info

Product

Resources

About