Discordant results among major histocompatibility complex binding affinity prediction tools

Abstract

Background: Human leukocyte antigen (HLA) alleles are critical components of the immune system’s ability to recognize and eliminate tumors and infections. A large number of machine learning-based major histocompatibility complex (MHC) binding affinity (BA) prediction tools have been developed and are widely used for both investigational and therapeutic applications, so it is important to explore differences in tool outputs. Methods: We examined predictions of four popular tools (netMHCpan, HLAthena, MHCflurry, and MHCnuggets) across a range of possible peptide sources (human, viral, and randomly generated) and MHC class I alleles. Results: We uncovered inconsistencies in predictions of BA, allele promiscuity and the relationship between physical properties of peptides by source and BA predictions, as well as quality of training data. We found amount of training data does not explain inconsistencies between tools and yet for all tools, predicted binding quantities are similar between human and viral proteomes. Lastly, we find peptide physical properties are associated with allele-specific binding predictions. Conclusions: Our work raises fundamental questions about the fidelity of peptide-MHC binding prediction tools and their real-world implications. The real-world use of these prediction tools for theoretical binding of peptides to alleles is worrying, as the range of allele promiscuity is substantial yet does not differentiate between potential foreign versus self-antigens. Evaluating more viruses – as well as bacteria, fungi, and other pathogens – and linking these analyses with metrics such as evolutionary distance may give greater insight into the relationship between HLA evolution and disease.

Improving generalizability for MHC-I binding peptide predictions through geometric deep learning

“…Moreover, these methods typically train and test on a subset of MHC alleles, which is minimal if compared to the broader spectrum of approximately 20,000 human alleles. Consequently, predictions for underrepresented alleles in these sets may exhibit lower accuracy 12 , akin to an out-of-distribution (OOD) issue in machine learning, where real-case scenarios differ significantly from training data, posing challenges in generalization [13][14][15] .…”

“…While these approaches have shown considerable advancement and contributions to clinical trials 14,15 , they are not without limitations and have been shown to provide highly discordant predictions 16,17 . A notable constraint stems from their reliance on extensive data, posing challenges for the thousands of less-explored HLA alleles with limited available binding information.…”