Groundwork for AI: Enforcing a benchmark for neoantigen prediction in personalized cancer immunotherapy

Abstract

This article expands on recent studies of machine learning or artificial intelligence (AI) algorithmsthat crucially depend on benchmark datasets, often called ‘ground truths.’ These ground-truth datasets gather input-data and output-targets, thereby establishing what can be retrieved computationally and evaluated statistically. I explore the case of the Tumor nEoantigen SeLection Alliance (TESLA), a consortium-based ground-truthing project in personalized cancer immunotherapy, where the ‘truth’ of the targets—immunogenic neoantigens—to be retrieved by the would-be AI algorithms depended on a broad technoscientific network whose setting up implied important organizational and material infrastructures. The study shows that instead of grounding an undisputable ‘truth’, the TESLA endeavor ended up establishing a contestable reference, the biology of neoantigens and how to measure their immunogenicity having slightly evolved alongside this four-year project. However, even if this controversy played down the scope of the TESLA ground truth, it did not discredit the whole undertaking. The magnitude of the technoscientific efforts that the TESLA project set into motion and the needs it ultimately succeeded in filling for the scientific and industrial community counterbalanced its metrological uncertainties, effectively instituting its contestable representation of ‘true’ neoantigens within the field of personalized cancer immunotherapy (at least temporarily). More generally, this case study indicates that the enforcement of ground truths, and what it leaves out, is a necessary condition to enable AI technologies in personalized medicine.

Enabling ‘AI’? The situated production of commensurabilities

“…Finally, another approach to the institutional rooting of ‘AI’ is offered by Jaton (2023) whose article furthers a line of research on the construction processes of ground-truth datasets—referential repositories that allow to train would-be ‘AI’ models and evaluate their performances—for the specialized case of personalized cancer immunotherapy. By retracing the history of the setting up of a challenge for the machine-learned detection of promising molecules (neoantigens), Jaton shows how the strategic assertion of an ‘AI-enabled cancer immunotherapy’ relies on the enforcement of benchmarked datasets, which are often costly and time-consuming, but also limited, contingent, and problematic, due in part to their readiness to create lock-in situations.…”