Why You Can’t Model Away Bias

Bode, Katherine

doi:10.1215/00267929-7933102

“…This approach bears out Katherine Bode's call for centering the "real epistemic, ethical, and political challenges" of constructing DH data sets and, more broadly, of "building a field in which humanities and statistical ways of knowing coexist and enhance each other." 38 It also iterates D'Ignazio and Klein's argument for "participatory, inclusive processes of knowledge production" in DH work. 39 This approach further performs intersectional feminist commitments to theorizing individual experience in dialogue with collective understanding, as it allows us to observe textual patterns across many readers' prose artifacts and then synthesize this larger-scale cultural analysis with attention to individual readers' responses.…”

Section: Subcorpusmentioning

confidence: 74%

The Generative Dissensus of Reading the Feminist Novel, 1995-2020: A Computational Analysis of Interpretive Communities

Mendelman

¹

,

Mukamal

²

2021

Journal of Cultural Analytics

View full text Add to dashboard Cite

This article furthers ongoing work on the merits of the feminist novel’s intrinsic variability by probing its dynamics in four publishing contexts: contemporary anglophone literary criticism, prestigious review publications, marketing materials, and online book reviews by social readers. We explore how these interpretive communities converge and diverge in their assessments of feminist fiction over the past twenty-five years by evaluating articles from the MLA International Bibliography, book reviews in The New York Times, The New Yorker, Times Literary Supp-lement, and other prominent periodicals, blurbs from Amazon, and Goodreads reviews. We trace the feminist novel’s ambivalent fates—or rather, feminist novels’ ambivalent fates—in and across these four domains. To do so, we engage computational methods of topic modeling, most distinctive word analysis, and named entity recognition. We synthesize these quantitative results with qualitative attention to provocative examples from our corpus. In so doing, we consider how literary scholars can develop more robust understandings of what feminism and feminist fiction mean to contemporary readers and what we stand to gain by bringing this diverse interpretive labor into our scholarly conversations.

show abstract

“…For one, this paper contributes to discussions of the importance of determining the nature of errors in a given body of textual data to understand their potential effects on scientific inquiry (Franzosi 2004). It also contributes to a growing literature on the consequential decisions taken in curation and preprocessing prior to analysis of corpora (Barberá et al 2021;Björk 2015;Bode 2020;Denny and Spirling 2018;Geiger et al 2020;Jo and Gebru 2020;Nguyen et al 2020;Schofield, Magnusson, and Mimno 2017;Sen et al 2019;Tahmasebi and Hengchen 2019). Moreover, it contributes to ongoing discussions bridging the oft-proclaimed chasm between qualitative and quantitative methods (Olsen 2004).…”

Section: Introductionmentioning

confidence: 94%

“…If data for integral parts of a social world are non-existent, this may lead to a sizeable coverage error because essential dimensions of the cultural world are unavailable for analysis (Dobson and Ziemann 2020;Mohr et al 2020;Pechenick, Danforth, and Dodds 2015). For example, surviving historical records in archives and libraries typically concern the wealthy, the educated, and the victors of war-unrepresentive of most of the populations and thus possibly hindering insights into the general target population (Bode 2020;Harris 2002;Mohr et al 2020). This error hence also relates to the concept of source criticism, and its call to consider the intent behind and the authenticity of historical documents (Dobson and Ziemann 2020).…”

Section: Research Inference Errorsmentioning

confidence: 99%

From Documents to Data: A Framework for Total Corpus Quality

Bodell¹,

Magnusson²,

Mützel³

2022

Preprint

2

0

View full text Add to dashboard Cite

As large corpora of digitized text and novel methodologies become increasingly available, researchers are rediscovering textual data’s potential fruitfulness for inquiries into social and cultural phenomena. While textual corpora show great promise to enrich our knowledge of the social, avoiding problems related to data quality remains a challenge to related empirical research. Hence, evaluating the quality of a corpus will be pivotal for future social science inquiries. We propose a conceptual framework for total corpus quality incorporating three important dimensions—total corpus error, corpus comparability, and corpus reproducibility—impacting the validity and reliability of inferences drawn from textual data. This framework provides insights toward evaluating and improving studies based on large-scale textual analyses. We employ a case study to exemplify and discuss how researchers can identify and measure the three proposed quality dimensions for any given corpus.

show abstract

“…If data for integral parts of a social world are non-existent, this may lead to a sizeable coverage error because essential dimensions of the cultural world are unavailable for analysis (Dobson and Ziemann 2020;Mohr et al 2020;Pechenick, Danforth, and Dodds 2015). For example, surviving historical records in archives and libraries typically concern the wealthy, the educated, and the victors of war-unrepresentive of most of the populations and thus possibly hindering insights into the general target population (Bode 2020;Harris 2002;Mohr et al 2020). This error hence also relates to the concept of source criticism, and its call to consider the intent behind and the authenticity of historical documents (Dobson and Ziemann 2020).…”

Section: Research Inference Errorsmentioning

confidence: 99%

“…For one, this paper contributes to discussions of the importance of determining the nature of errors in a given body of textual data to understand their potential effects on scientific inquiry (Franzosi 2004). It also contributes to a growing literature on the consequential decisions taken in curation and preprocessing prior to analysis of corpora (Barberá et al 2021;Björk 2015;Bode 2020;Denny and Spirling 2018;Geiger et al 2020;Jo and Gebru 2020;Nguyen et al 2020;Schofield, Magnusson, and Mimno 2017;Sen et al 2019;Tahmasebi and Hengchen 2019). Moreover, it contributes to ongoing discussions bridging the oft-proclaimed chasm between qualitative and quantitative methods (Olsen 2004).…”

Section: Introductionmentioning

confidence: 95%

From Documents to Data: A Framework for Total Corpus Quality

Bodell

¹

,

Magnusson

²

,

Mützel

³

2022

Socius: Sociological Research for a Dynamic World

View full text Add to dashboard Cite

As large corpora of digitized text become increasingly available, researchers are rediscovering textual data’s potential fruitfulness for inquiries into social and cultural phenomena. Although textual corpora promise to enrich our knowledge of the social world, avoiding problems related to data quality remains a challenge to related empirical research. Hence, evaluating the quality of a corpus will be pivotal for future social scientific inquiries. The authors propose a conceptual framework for total corpus quality, incorporating three crucial dimensions: total corpus error, corpus comparability, and corpus reproducibility. These dimensions affect the validity and reliability of inferences drawn from textual data. In addition, the authors’ framework provides insights toward evaluating and improving studies on the basis of large-scale textual analyses. After outlining this framework, the authors then illustrate an application of the total corpus quality framework by an example case study using digitized newspaper articles to study topic salience over 75 years.

show abstract

Why You Can’t Model Away Bias

Cited by 34 publications

References 43 publications

The Generative Dissensus of Reading the Feminist Novel, 1995-2020: A Computational Analysis of Interpretive Communities

The Generative Dissensus of Reading the Feminist Novel, 1995-2020: A Computational Analysis of Interpretive Communities

From Documents to Data: A Framework for Total Corpus Quality

From Documents to Data: A Framework for Total Corpus Quality

Contact Info

Product

Resources

About