Predicting SARS-CoV-2 epitope-specific TCR recognition using pre-trained protein embeddings

Abstract

The COVID-19 pandemic is ongoing because of the high transmission rate and the emergence of SARS-CoV-2 variants. The P272L mutation in SARS-Cov-2 S-protein is known to be highly relevant to the viral escape associated with the second pandemic wave in Europe. Epitope-specific T-cell receptor (TCR) recognition is a key factor in determining the T-cell immunogenicity of a SARS-CoV-2 epitope. Although several data-driven methods for predicting epitope-specific TCR recognition have been proposed, they remain challenging owing to the enormous diversity of TCRs and the lack of available training data. Self-supervised transfer learning has recently been demonstrated to be powerful for extracting useful information from unlabeled protein sequences and increasing the predictive performance of the fine-tuned models in downstream tasks.Here, we present a predictive model based on Bidirectional Encoder Representations from Transformers (BERT), employing self-supervised transfer learning, to predict SARS-CoV-2 T-cell epitope-specific TCR recognition. The fine-tuned model showed notably high predictive performance for independent evaluation using the SARS-CoV-2 epitope-specific TCR CDR3β sequence datasets. In particular, we found the proline at position 4 corresponding to the P272L mutation in the SARS-CoV-2 S-protein269-277 epitope (YLQPRTFLL) may contribute substantially to TCR recognition of the epitope through interpreting the output attention weights of our model.We anticipate that our findings will provide new directions for constructing a reliable data-driven model to predict the immunogenic T-cell epitopes using limited training data and help accelerate the development of an effective vaccine in response to SARS-CoV-2 variants.

De novogeneration of T-cell receptors with desired epitope-binding property by leveraging a pre-trained large language model

“…Recently, the birth of Bidirectional Encoder Representations from Transformers (BERT) models revolutionized the natural language processing (NLP) field 18 . Because of the similarities between protein sequences and language sentences, several studies applied BERT models to TCR-related tasks 19,20 . TCR-BERT 19 utilized over 8,000 TCR sequences on the self-supervised learning tasks, and showed promising performance on the downstream tasks, such as TCR-antigen binding affinity prediction and sequence clustering.…”

“…TCR-BERT 19 utilized over 8,000 TCR sequences on the self-supervised learning tasks, and showed promising performance on the downstream tasks, such as TCR-antigen binding affinity prediction and sequence clustering. Han et al 20 presented a BERT-based model by fine-tuning the pre-trained Tasks Assessing Protein Embeddings (TAPE) 21 model to predict SARS-CoV-2 T-cell epitope-specific TCR recognition. However, to the best of our knowledge, all the existing approaches only focused on the modeling of natural TCR sequences.…”