Incorporating Dialectal Variability for Socially Equitable Language
            Identification

Jurgens, David; Tsvetkov, Yulia; Jurafsky, Dan

doi:10.18653/v1/p17-2009

Cited by 61 publications

(50 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The results prove that BERT expresses strong preferences for male pronouns, raising concerns with using BERT in downstream tasks like resume filtering. Table 5: Percentage of attributes associated more strongly with the male gender 6 Related Work NLP applications ranging from core tasks such as coreference resolution (Rudinger et al, 2018) and language identification (Jurgens et al, 2017), to downstream systems such as automated essay scoring (Amorim et al, 2018), exhibit inherent social biases which are attributed to the datasets used to train the embeddings (Barocas and Selbst, 2016;Zhao et al, 2017;Yao and Huang, 2017).…”

Section: Real World Implicationsmentioning

confidence: 99%

Measuring Bias in Contextualized Word Representations

Kurita¹,

Vyas²,

Pareek³

et al. 2019

Proceedings of the First Workshop on Gender Bias in Natural Language Processing

Self Cite

294

275

View full text Add to dashboard Cite

Contextual word embeddings such as BERT have achieved state of the art performance in numerous NLP tasks. Since they are optimized to capture the statistical properties of training data, they tend to pick up on and amplify social stereotypes present in the data as well. In this study, we (1) propose a template-based method to quantify bias in BERT;(2) show that this method obtains more consistent results in capturing social biases than the traditional cosine based method; and (3) conduct a case study, evaluating gender bias in a downstream task of Gender Pronoun Resolution. Although our case study focuses on gender bias, the proposed technique is generalizable to unveiling other biases, including in multiclass settings, such as racial and religious biases.

show abstract

Section: Real World Implicationsmentioning

confidence: 99%

Measuring Bias in Contextualized Word Representations

Kurita¹,

Vyas²,

Pareek³

et al. 2019

Proceedings of the First Workshop on Gender Bias in Natural Language Processing

Self Cite

294

275

View full text Add to dashboard Cite

show abstract

“…We primarily evaluate on the task of language identification ("LangID": Cavnar and Trenkle (1994)), using the corpora of Lui and Baldwin (2012), which combine large training sets over a diverse range of text domains. Domain adaptation is an important problem for this task (Lui and Baldwin, 2014;Jurgens et al, 2017), where text resources are collected from numerous sources, and exhibit a wide variety of language use. We show that while domain adversarial training overall improves over baselines, gains are modest.…”

Section: Introductionmentioning

confidence: 99%

What’s in a Domain? Learning Domain-Robust Text Representations using Adversarial Training

Baldwin

Cohn

2018

Proceedings of the 2018 Conference of the North American Chapter Of the Association for Computational Linguistics: Hu

View full text Add to dashboard Cite

Most real world language problems require learning from heterogenous corpora, raising the problem of learning robust models which generalise well to both similar (in domain) and dissimilar (out of domain) instances to those seen in training. This requires learning an underlying task, while not learning irrelevant signals and biases specific to individual domains. We propose a novel method to optimise both in-and out-of-domain accuracy based on joint learning of a structured neural model with domain-specific and domain-general components, coupled with adversarial training for domain. Evaluating on multi-domain language identification and multi-domain sentiment analysis, we show substantial improvements over standard domain adaptation techniques, and domain-adversarial training.

show abstract

“…used byte-level representations of sentences as input for the networks.Recently, Hanani et al (2016) and also used LSTMs. Later, GRUs were successfully used for LI byJurgens et al (2017) andKocmi and Bojar (2017). In addition to GRUs,Bjerva (2016) also experimented with deep residual networks ("ResNets") at DSL 2016.…”

mentioning

confidence: 99%

Automatic Language Identification in Texts: A Survey

Jauhiainen

Lui²,

Zampieri³

et al. 2019

jair

104

View full text Add to dashboard Cite

Language identification ("LI") is the problem of determining the natural language that a document or part thereof is written in. Automatic LI has been extensively researched for over fifty years. Today, LI is a key part of many text processing pipelines, as text processing techniques generally assume that the language of the input text is known. Research in this area has recently been especially active. This article provides a brief history of LI research, and an extensive survey of the features and methods used in the LI literature. We describe the features and methods using a unified notation, to make the relationships between methods clearer. We discuss evaluation methods, applications of LI, as well as off-the-shelf LI systems that do not require training by the end user. Finally, we identify open issues, survey the work to date on each issue, and propose future directions for research in LI.LI as a task predates computational methods -the earliest interest in the area was motivated by the needs of translators, and simple manual methods were developed to quickly identify documents in specific languages. The earliest known work to describe a functional LI program for text is by Mustonen (1965), a statistician, who used multiple discriminant analysis to teach a computer how to distinguish, at the word level, between English, Swedish and Finnish. Mustonen compiled a list of linguistically-motivated character-based features, and trained his language identifier on 300 words for each of the three target languages. The training procedure created two discriminant functions, which were tested with 100 words for each language. The experiment resulted in 76% of the words being correctly classified; even by current standards this percentage would be seen as acceptable given the small amount of training material, although the composition of training and test data is not clear, making the experiment unreproducible.In the early 1970s, Nakamura (1971) considered the problem of automatic LI. According to Rau (1974) and the available abstract of Nakamura's article, 1 his language identifier was able to distinguish between 25 languages written with the Latin alphabet. As features, the method used the occurrence rates of characters and words in each language. From the abstract it seems that, in addition to the frequencies, he used some binary presence/absence features of particular characters or words, based on manual LI. Rau (1974) wrote his master's thesis "Language Identification by Statistical Analysis" for the Naval Postgraduate School at Monterey, California. The continued interest and the need to use LI of text in military intelligence settings is evidenced by the recent articles of, for example, Rafidha Rehiman et al. (2013), Rowe et al. (2013), and Voss et al. (2014. As features for LI, Rau (1974) used, e.g., the relative frequencies of characters and character bigrams. With a majority vote classifier ensemble of seven classifiers using Kolmogor-Smirnov's Test of Goodness of Fit and Yule's characteristic (K), he managed...

show abstract

Incorporating Dialectal Variability for Socially Equitable Language Identification

Cited by 61 publications

References 31 publications

Measuring Bias in Contextualized Word Representations

Measuring Bias in Contextualized Word Representations

What’s in a Domain? Learning Domain-Robust Text Representations using Adversarial Training

Automatic Language Identification in Texts: A Survey

Contact Info

Product

Resources

About