On reading and interpreting black box deep neural networks

Abstract

The deep neural networks used in computer vision and in recent large language models are widely recognized as black boxes, a term that describes their complicated architectures and opaque decision-making mechanisms. This essay outlines several different strategies through which humanist researchers and critics of machine learning might better understand and interpret the class of deep learning methods known as Transformers. These strategies expose different aspects of what might be “learned” as Transformers are trained and used in the analysis of language and can help critics at least partially open the black box of machine learning. They are also especially useful for digital humanists using these models as part of a research program informed by tool criticism in which the use of computational tools is conceived of as a metainterpretive act.

Reproducibility and explainability in digital humanities

“…Already in 2017, Doshi-Velez and Kim had put forward the goal of building a "rigorous science of interpretable machine learning" (Doshi-Velez & Kim, 2017). Since then, countless papers and projects have contributed to an ever-growing pool of approaches, aiming at text and image models alike (see (Molnar, 2020), or the contribution by James Dobson in this issue (Dobson, 2023) for an overview of the most popular techniques). At the same time, the question of exposing a model's particular perspective on the world, that is, describing its biases and limitations, is, at its core, a humanities question.…”

“…The contributions to this special issue, together, make a significant leap towards the methodological and epistemological reflection of such methods, a reflection that so far has been missing in the digital humanities context. James Dobson, in "Reading and Interpreting Black Box Deep Neural Networks", gives an in-depth introduction to interpretability and explainability methods from computer science research, arguing for a "research program informed by tool criticism in which the use of computational tools is conceived of as a metainterpretive act" (Dobson, 2023). Tobias Blanke, Tommaso Venturini, and Kari De Pryck demonstrate how political concepts like "leadership", which are difficult to operationalize, can be better approximated through XAI techniques (Blanke et al, 2023).…”

“…This special issue, Reproducibility and Explainability in Digital Humanities, was published in two versions: a print version (International Journal of Digital Humanities, 2023a;Burrows, 2023;El-Hajj et al, 2023;Justin & Menon, 2023;Middle, 2023;Schöch, 2023;Siddiqui, 2023;Dobson, 2023;Pandiani et al, 2023;Rudman, 2023;Chun & Elkins, 2023), and an extended online special collection (International Journal of Digital Humanities, 2023b) that features all articles of the print version (a number of them as gold open access publications) plus five additional articles (Joyeux-Prunel, 2023;Huskey, 2023;Blanke et al, 2023;Cooke & Litvack-Katzman, 2023;Hankins, 2023).…”