Curated Data In — Trustworthy In Silico Models Out: The Impact of Data Quality on the Reliability of Artificial Intelligence Models as Alternatives to Animal Testing

Abstract

New Approach Methodologies (NAMs) that employ artificial intelligence (AI) for predicting adverse effects of chemicals have generated optimistic expectations as alternatives to animal testing. However, the major underappreciated challenge in developing robust and predictive AI models is the impact of the quality of the input data on the model accuracy. Indeed, poor data reproducibility and quality have been frequently cited as factors contributing to the crisis in biomedical research, as well as similar shortcomings in the fields of toxicology and chemistry. In this article, we review the most recent efforts to improve confidence in the robustness of toxicological data and investigate the impact that data curation has on the confidence in model predictions. We also present two case studies demonstrating the effect of data curation on the performance of AI models for predicting skin sensitisation and skin irritation. We show that, whereas models generated with uncurated data had a 7–24% higher correct classification rate (CCR), the perceived performance was, in fact, inflated owing to the high number of duplicates in the training set. We assert that data curation is a critical step in building computational models, to help ensure that reliable predictions of chemical toxicity are achieved through use of the models.

AI ethics: the case for including animals

“…Lethal autonomous weapons that target humans have led to much debate within and beyond the field of AI ethics. 29 But drones also target animals, especially those who are deemed by some humans to be "invasive" or "pests. "…”

“…As already noted, AI systems that can predict the toxicity of novel chemicals are being developed. Such technologies have the potential to eliminate procedures that currently cause the death of millions of animals every year, often after prolonged suffering [29]. Many countries do not keep statistics on the number of animals used in toxicity testing, but in the European Union, the number of animals used for this purpose in the most recent year for which figures are currently available was approximately 850,000 [30].…”

“…alamy. com/ stock-image-crush ed-frog-on-the-road-thedeath-of-anima ls-on-the-roads-16354 6123. html 29. Future of Life Institute.…”

STopTox: An in Silico Alternative to Animal Testing for Acute Systemic and Topical Toxicity

“…Previously, we already expressed concerns that the high accuracy of models as reported ( Luechtefeld et al. 2018 ) could be the consequence of inadequate data curation, leaving many replicate compounds in the modeling and validation data sets ( Alves et al. 2021 ).…”

Clean and Safe Drinking Water Systems via Metagenomics Data and Artificial Intelligence: State-of-the-Art and Future Perspective

“…Thus, these hard-to-classify sequences are most likely indicative of novel lineages which are characteristic of the drinking water microbiome and may play vital roles in drinking water biogeochemical processes (Revetta et al, 2010). As a consequence of this knowledge gap, light must be shed on the limitations of any artificial-intelligence-based models that use metagenomics data because the performance of any data-driven mathematical model depends on the quality of data it is fed (Sessions and Valtorta, 2006;Alves et al, 2021;Sambasivan et al, 2021).…”