Embeddings from deep learning transfer GO annotations beyond homology

Abstract

Knowing protein function is crucial to advance molecular and medical biology, yet experimental function annotations through the Gene Ontology (GO) exist for fewer than 0.5% of all known proteins. Computational methods bridge this sequence-annotation gap typically through homology-based annotation transfer by identifying sequence-similar proteins with known function or through prediction methods using evolutionary information. Here, we propose predicting GO terms through annotation transfer based on proximity of proteins in the SeqVec embedding rather than in sequence space. These embeddings originate from deep learned language models (LMs) for protein sequences (SeqVec) transferring the knowledge gained from predicting the next amino acid in 33 million protein sequences. Replicating the conditions of CAFA3, our method reaches an Fmax of 37 ± 2%, 50 ± 3%, and 57 ± 2% for BPO, MFO, and CCO, respectively. Numerically, this appears close to the top ten CAFA3 methods. When restricting the annotation transfer to proteins with < 20% pairwise sequence identity to the query, performance drops (Fmax BPO 33 ± 2%, MFO 43 ± 3%, CCO 53 ± 2%); this still outperforms naïve sequence-based transfer. Preliminary results from CAFA4 appear to confirm these findings. Overall, this new concept is likely to change the annotation of proteins, in particular for proteins from smaller families or proteins with intrinsically disordered regions.

Embeddings from protein language models predict conservation and variant effects

“…Embeddings have been used successfully as exclusive input to predicting 6 secondary structure and subcellular localization at performance levels almost reaching (Alley et al 72019; Heinzinger et al 2019;Rives et al 2021) or even exceeding (Elnaggar et al 2021;Stärk et al 8 2021) state-of-the-art methods using evolutionary information from MSAs as input. Embeddings can 9 even substitute sequence similarity for homology-based annotation transfer (Littmann et al 2021a;Littmann et al 2021b). The power of such embeddings has been increasing with the advance of algorithms (Elnaggar et al 2021).…”

Learned Embeddings from Deep Learning to Visualize and Predict Protein Sets

“…Previously, machine-learning methods in computational biology leveraged data-driven protein representations such as substitution matrices, capturing biophysical features (Henikoff & Henikoff, 1992), family-specific profiles (Stormo et al, 1982), or evolutionary couplings (Morcos et al, 2011) that capture evolutionary features. Now, embeddings provide competitive results for many prediction tasks (Littmann et al, 2021;Rao et al, 2019Rao et al, , 2020. Protein LMs may even be combined with other representations to gain even better performance (Rives et al, 2019;Villegas-Morcillo et al, 2020).…”

“…Via the "extract" stage, the pipeline incorporates supervised and unsupervised approaches for protein embeddings to further enhance analytical potential out-ofthe-box. For instance, users can extract secondary structure in 3-and 8-states for embeddings from SeqVec (Heinzinger et al, 2019) and ProtBert (Elnaggar et al, 2020), or transfer GO annotations using embeddings of any available LM (Littmann et al, 2021). Pipeline runs are reproducible, as configurations are defined through files, and the output is stored in easily exchangeable formats, e.g., CSVs, FASTA, and HDF5 (The HDF Group, 2000).…”

“…Although embedding-based predictions tend to be less accurate than those using evolutionary information, they require less time (Heinzinger et al, 2019;Rao et al, 2019;Rives et al, 2019). By learning to represent sequence and background information, embeddings open the door to a completely new way of using protein sequences, successful enough to even compete with traditional remote homology detection and structural alignments (Biasini et al, 2014;Littmann, Heinzinger, Dallago, Olenyi, & Rost, 2021;Morton et al, 2020;Villegas-Morcillo et al, 2020).…”

Computational Identification of Functional Centers in Complex Proteins: A Step-by-Step Guide With Examples

“…Furthermore, the method enables candidate selections for follow-up in vivo and in planta studies that will eventually reveal the biological roles of these functional centers such as those reported in Joudoi et al (2013), Shen et al (2019), Vaz Dias et al (2019, Angkawijaya et al (2020), Lee et al (2020), andTurek et al (2020). We foresee that emerging experimental data will inform and strengthen motif refinement efforts, and enable the development of modern machine learning techniques that incorporate multiple features ranging from the classical physicochemical properties of protein domains and protein-protein interaction (PPI) networks to GO based function predictions, to not only automate annotations for uncharacterized proteins, but also to identify hidden functional centers in complex multi-functional proteins (Rifaioglu et al, 2019;Bonetta and Valentino, 2020;Cai et al, 2020;Littmann et al, 2021).…”