Do we know what we are doing?

Abstract

In November 2012, the newly created Open Science Collaboration published a brief article announcing a multi-year effort to "estimate the reproducibility of psychological science." The collaboration was directed by Brian Nosek of the University of Virginia and would eventually involve over 250 co-authors. According to the collaboration, reproducibility was one of, if not the single most defining feature of the social endeavor known as "science." "Other types of belief," the authors write, "depend on the authority and motivations of the source; beliefs in science do not." The ability to reproduce scientific results across time and space-the ability to have results be independent of the individuals involved-is what the authors argued makes science science. And yet the eventual findings of the reproducibility project showed a remarkable reproductive failure. Over half of all studies failed to indicate similar effects upon replication. The very value upon which science was supposed to be founded appeared to be an exception rather than a norm. In November 2012, the newly created Open Science Collaboration published a brief article announcing a multi-year effort to "estimate the reproducibility of psychological science." 1 The collaboration was directed by Brian Nosek of the University of Virginia and would eventually involve over 250 co-authors. According to the collaboration, reproducibility was one of, if not the single most defining feature of the social endeavor known as "science." "Other types of belief," the authors write, "depend on the authority and motivations of the source; beliefs in science do not." 2 The ability to reproduce scientific results across time and space-the ability to have results be independent of the individuals involved-is what the authors argued makes science science. And yet the eventual findings of the reproducibility project showed a remarkable reproductive failure. 3 Over half of all studies failed to indicate similar effects upon replication. 4 The very value upon which science was supposed to be founded appeared to be an exception rather than a norm. 5 Nan Z. Da's study published in Critical Inquiry is part of a growing body of work that seeks to introduce the idea of replication into the humanities.

The General Linear Model and Minimal Research Compendiums: An Approach to Assure Statistical Validity in Digital Humanities Research

“…For instance, in her article, Da often misrepresented simple statistical concepts such as p-values, confidence intervals, and alpha cutoffs (N. Z. Da, 2020;Piper, 2020;Underwood, 2020;Weatherby, 2020). She later admitted that she did not see herself as an expert on the topic and that an outside reviewer had questioned her findings.…”

Unlikely Load Lifters

“…I would define it as unmerited love or courtesy. Piper (2020) described grace as “undeserved favor” or “action or the power or the influence or the force of this disposition, which produces real, practical outcomes in people's lives.” Zahl (2007) called it “love that seeks you out when you have nothing to give in return. Grace is love coming at you that has nothing to do with you.…”

Tool Criticism and the Computational Turn. A “Methodological Moment” in Media and Communication Studies

“…From the informative disciplinary dispute between Nan Z. Da and Andrew Piper over the role of quantitative methods in the humanities, we can identify a number of misperceptions about digital methods that relate to these criticisms. Whereas Da (2019) provides cautionary tales and offers a dismissive take on computational literary studies, Piper (2019) fires back and underscores the rich opportunities.…”