Reproducibility, verifiability, and computational historical research

Abstract

Digital humanities methods have been at the heart of a recent series of high-profile historical research projects. But these approaches raise new questions about reproducibility and verifiability in a field of research where grounding one’s conclusions in a body of historical evidence is crucial. While there have been extensive debates about the nature and methods of historical research since the nineteenth century, the underlying assumption has generally been that documenting one’s sources in a series of footnotes is essential to enable other researchers to test the validity of the research. Even if this approach never amounted to “reproducibility” in the sense of scientific experimentation, it might still be seen as broadly analogous, since the evidence can be reassembled to see the basis for the explanations that were offered and to test their validity. This essay examines how new digital methods like topic modelling, network analysis, knowledge graphs, species models, and various kinds of visualizations are affecting the process of reproducing and verifying historical research. Using examples drawn from recent research projects, it identifies a need for thorough documentation and publication of the different layers of digital research: digital and digitized collections, descriptive metadata, the software used for analysis and visualizations, and the various settings and configurations.

Reproducibility and explainability in digital humanities

“…He also explores the methodological building blocks needed to meet these standards (Rudman, 2023). Toby Burrows critically explores the verifiability of sources, and considers the impact of digital methods on reproducibility in historical research in general (Burrows, 2023). Gabriel Hankins investigates issues of reproducibility regarding archival silences and biases in data sets in the "struggle over the autonomy of disciplinary stan-dards of argumentation and evidence" while suggesting a practice of "data readings" in his case study of the Major [literary] Prizes Database (Hankins, 2023).…”

“…This special issue, Reproducibility and Explainability in Digital Humanities, was published in two versions: a print version (International Journal of Digital Humanities, 2023a;Burrows, 2023;El-Hajj et al, 2023;Justin & Menon, 2023;Middle, 2023;Schöch, 2023;Siddiqui, 2023;Dobson, 2023;Pandiani et al, 2023;Rudman, 2023;Chun & Elkins, 2023), and an extended online special collection (International Journal of Digital Humanities, 2023b) that features all articles of the print version (a number of them as gold open access publications) plus five additional articles (Joyeux-Prunel, 2023;Huskey, 2023;Blanke et al, 2023;Cooke & Litvack-Katzman, 2023;Hankins, 2023).…”