Closing the gap between measurable genetic information and observable traits is a longstanding challenge in genomics. Yet, the prediction of molecular phenotypes from DNA sequences alone remains limited and inaccurate, often driven by the scarcity of annotated data and the inability to transfer learnings between prediction tasks. Here, we present an extensive study of foundation models pre-trained on DNA sequences, named the Nucleotide Transformer, integrating information from 3,202 diverse human genomes, as well as 850 genomes from a wide range of species, including model and non-model organisms. These transformer models yield transferable, context-specific representations of nucleotide sequences, which allow for accurate molecular phenotype prediction even in low-data settings. We show that the representations alone match or outperform specialized methods on 11 of 18 prediction tasks, and up to 15 after fine-tuning. Despite no supervision, the transformer models learnt to focus attention on key genomic elements, including those that regulate gene expression, such as enhancers. Lastly, we demonstrate that utilizing model representations alone can improve the prioritization of functional genetic variants. The training and application of foundational models in genomics explored in this study provide a widely applicable stepping stone to bridge the gap of accurate molecular phenotype prediction from DNA sequence alone.

No abstract

The ongoing COVID-19 pandemic is leading to the discovery of hundreds of novel SARS-CoV-2 variants on a daily basis. While most variants do not impact the course of the pandemic, some variants pose significantly increased risk when the acquired mutations allow better evasion of antibody neutralisation in previously infected or vaccinated subjects, or increased transmissibility. Early detection of such high risk variants (HRVs) is paramount for the proper management of the pandemic. However, experimental assays to determine immune evasion and transmissibility characteristics of new variants are resource-intensive and time-consuming, potentially leading to delayed appropriate responses by decision makers. Here we present a novel in silico approach combining Spike protein structure modelling and large protein transformer language models on Spike protein sequences, to accurately rank SARS-CoV-2 variants for immune escape and fitness potential. We validate our immune escape and fitness metrics with in vitro pVNT and binding assays. These metrics can be combined into an automated Early Warning System (EWS) capable of evaluating new variants in minutes and risk monitoring variant lineages in near real-time. The EWS flagged 12 out of 13 variants, designated by the World Health Organisation (WHO, Alpha-Omicron) as potentially dangerous, on average two months ahead of them being designated as such, demonstrating its ability to help increase preparedness against future variants. Omicron was flagged by the EWS on the day its sequence was made available, with immune evasion and binding metrics subsequently confirmed through our in vitro experiments.

No abstract

The continued evolution of the SARS-CoV-2 Omicron variant has led to the emergence of numerous sublineages with different patterns of evasion from neutralizing antibodies. We investigated neutralizing activity in immune sera from individuals vaccinated with SARS-CoV-2 wild-type spike (S) glycoprotein-based COVID-19 mRNA vaccines after subsequent breakthrough infection with Omicron BA.1, BA.2, or BA.4/BA.5 to study antibody responses against sublineages of high relevance. We report that exposure of vaccinated individuals to infections with Omicron sublineages, and especially with BA.4/BA.5, results in a boost of Omicron BA.4.6, BF.7, BQ.1.1, and BA.2.75 neutralization, but does not efficiently boost neutralization of sublineages BA.2.75.2 and XBB. Accordingly, we found in in silico analyses that with occurrence of the Omicron lineage a large portion of neutralizing B-cell epitopes were lost, and that in Omicron BA.2.75.2 and XBB less than 12% of the wild-type strain epitopes are conserved. In contrast, HLA class I and class II presented T-cell epitopes in the S glycoprotein were highly conserved across the entire evolution of SARS-CoV-2 including Alpha, Beta, and Delta and Omicron sublineages, suggesting that CD8+ and CD4+ T-cell recognition of Omicron BQ.1.1, BA.2.75.2, and XBB may be largely intact. Our study suggests that while some Omicron sublineages effectively evade B-cell immunity by altering neutralizing antibody epitopes, S protein-specific T-cell immunity, due to the very nature of the polymorphic cell-mediated immune, response is likely to remain unimpacted and may continue to contribute to prevention or limitation of severe COVID-19 manifestation.