How We Do Things With Words: Analyzing Text as Social and Cultural Data

Abstract

In this article we describe our experiences with computational text analysis involving rich social and cultural concepts. We hope to achieve three primary goals. First, we aim to shed light on thorny issues not always at the forefront of discussions about computational text analysis methods. Second, we hope to provide a set of key questions that can guide work in this area. Our guidance is based on our own experiences and is therefore inherently imperfect. Still, given our diversity of disciplinary backgrounds and research practices, we hope to capture a range of ideas and identify commonalities that resonate for many. This leads to our final goal: to help promote interdisciplinary collaborations. Interdisciplinary insights and partnerships are essential for realizing the full potential of any computational text analysis involving social and cultural concepts, and the more we bridge these divides, the more fruitful we believe our work will be.

Content Analysis of Textbooks via Natural Language Processing: Findings on Gender, Race, and Ethnicity in Texas U.S. History Textbooks

“…Measuring Connotations of Descriptors. Lexicon-based approaches illuminate the affective and social connotations of words, an area of great importance for the social sciences (Nguyen et al, 2019). This method counts the number of words occurring in a text that are defined in a lexicon as denoting a particular meaning, such as words of positive sentiment.…”

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Demonstrations

“…"Single-task" refers to as the single-task training approach. † denotes scores taken from the PhoBERT paper (Nguyen and Nguyen, 2020). Note that "Singletask" NER is not affected by the data leakage issue.…”

“…It has been boosted by the success of the national project on Vietnamese language and speech processing (VLSP) KC01.01/2006-2010 and VLSP workshops that have run shared tasks since 2013. 1 Fundamental tasks of POS tagging, NER and dependency parsing thus play important roles, providing useful features for many downstream application tasks such as machine translation (Tran et al, 2016), sentiment analysis (Bang and Sornlertlamvanich, 2018), relation extraction (To and Do, 2020), semantic parsing (Nguyen et al, 2020), open information extraction (Truong et al, 2017) and question answering (Nguyen et al, 2017;Le-Hong and Bui, 2018). Thus, there is a need to develop NLP toolkits for linguistic annotations w.r.t.…”

“…VnCoreNLP (Vu et al, 2018) is the previous public toolkit employing traditional featurebased machine learning models to handle those Vietnamese NLP tasks. However, VnCoreNLP is now no longer considered state-of-the-art because its performance results are significantly outperformed by ones obtained when fine-tuning PhoBERT-the current state-of-the-art monolingual pre-trained language model for Vietnamese (Nguyen and Nguyen, 2020). Note that there are no publicly available fine-tuned BERT-based models for the three Vietnamese tasks.…”

Expert Concept-Modeling Ground Truth Construction for Word Embeddings Evaluation in Concept-Focused Domains

“…The theoretical methodology behind their actual and concrete construction has, to the best of our ken, never been explicitly conceptualised and properly systematised. It is on the contrary rather customary to state that the notion of ground truth is inapplicable to the humanities and the social sciences (Nguyen et al (2020)), which are the domains in which theoretical reflection on textual representation of ideas and interpretation mostly tends to take place. Most work on historical concept change does not involve ground truths xc in evaluation (cf.…”