Disambiguation of morpho-syntactic features of African American English – the case of habitual be

Abstract

Recent research has highlighted that natural language processing (NLP) systems exhibit a bias against African American speakers. The bias errors are often caused by poor representation of linguistic features unique to African American English (AAE), due to the relatively low probability of occurrence of many such features in training data. We present a workflow to overcome such bias in the case of habitual "be". Habitual "be" is isomorphic, and therefore ambiguous, with other forms of "be" found in both AAE and other varieties of English. This creates a clear challenge for bias in NLP technologies. To overcome the scarcity, we employ a combination of rule-based filters and data augmentation that generate a corpus balanced between habitual and non-habitual instances. With this balanced corpus, we train unbiased machine learning classifiers, as demonstrated on a corpus of AAE transcribed texts, achieving .65 F 1 score disambiguating habitual "be".

Leveraging syntactic dependencies in disambiguation: the case of African American English

“…Recently, there has been a surge in NLP research for AAE. Studies have explored dependency parsing (Blodgett et al, 2018), POS-tagging (Dacon, 2022;Jørgensen et al, 2016), hate speech classification (Harris et al, 2022;Sap et al, 2019), automatic speech recognition (Koenecke et al, 2020;Martin and Tang, 2020), dialectal analysis (Blodgett et al, 2016;Dacon, 2022;Stewart, 2014) and feature detection (Masis et al, 2022;Santiago et al, 2022). Projects such as these rely heavily on large amounts of labeled data, however, little research is dedicated to optimizing the disambiguation and annotation process.…”

“…In their work, Santiago et al (2022) leveraged linguistic descriptions and POS tagging to develop a rule-based method for identifying many nonhabitual instances of "be". This allowed us to filter out these instances, thereby balancing our dataset by increasing the frequency of habitual be examples.…”

“…This allowed us to filter out these instances, thereby balancing our dataset by increasing the frequency of habitual be examples. Implementing this filtering method, we found that Santiago et al (2022) did not account for patterns of phonetic variation within the transcripts. As a result of this discovery, we modified their POS1 to include the words "wanna", "trynna", "gonna", and "gotta" in addition to the more standardized representations already covered by the filter ("want to", "trying to", "going to", "got to").…”

“…Because of this, we used scikit-learn (Pedregosa et al, 2011) to implement an ensemble model that combines logistic regression (LR), multilayer perceptron (MLP), and support vector classification (SVC) modes. This decision was also motivated by Santiago et al (2022) who demonstrated ensemble models performed better than individual models. The 'balanced' parameter was added to the LR and SVC models because the two classes were severely unbalanced; however, the MLP model has no such parameter.…”

“…The use of be as a habitual aspect marker in AAE presents challenges for disambiguation due to its infrequency and lexical similarly to more common uses of the word. Building on previous research that explored part-of-speech (POS) tagging as a disambiguation parameter (Santiago et al, 2022), our study delves deeper into the complex syntactic patterns co-occuring with habitual and non-habitual meanings of "be" in African American speech. Using linguistic data from two oral history corpora of AAE speakers, we demonstrate that integrating dependency parsing enhances automatic detection compared to the baseline and previous POS-only approaches.…”