Development of in silico methodologies to predict the toxicity of novel proteins in the context of food and feed risk assessment

Abstract

This report is the outcome of an EFSA procurement (OC/EFSA/GMO/2021/02 – LOT1) aiming at developing an in silico strategy to predict the toxicity of (novel) proteins. Up‐to‐date, commercially available tools predicting protein toxicity based on primary structures were evaluated for their accuracy and usability, using a curated dataset of annotated toxins and non‐toxins from UniProt. ToxinPred2 and Toxify emerged as the top performers, showing both high accuracy and suitability for integration into an automated pipeline. Additional bioinformatics methods were explored, which provide sequence similarity‐based information rather than direct predictions (BLAST, InterPro HMM profiles). By converting their outputs into features for machine learning models, a high prediction accuracy was achieved, though there is potential for improvement to reduce overfitting risks. An Artificial Intelligence (AI)‐based consensus pipeline, integrating results from ToxinPred2, Toxify, and our machine learning models was developed. This consensus model reached a 95% accuracy rate in distinguishing toxins from non‐toxins. Noteworthy, our BLAST‐based machine learning model ‐ although performance‐wise comparable to BLAST ‐ offers higher sensitivity and specificity across diverse queries than BLAST; it relies on database‐based evolutionary relationships, which may significantly limit its applicability to novel or mutated toxins. Structure‐based prediction methods are deemed impractical due to their resource intensity and reliance on accurate structural data; AI‐driven structure prediction methods ‐ like Rosetta and AlphaFold ‐ are promising, however they are still under development and may not be suitable for the regulatory context yet. Recommendations are provided, including enhancement of the proposed consensus pipeline to create an independent open‐source, user‐friendly tool for evaluating the safety of (novel) proteins in food and feed; regular updates of the proposed databases and models; incorporation of 3D structures and in general validation of AI and machine learning models for regulatory uses.