Using deep learning to annotate the protein universe

Abstract

Understanding the relationship between amino acid sequence and protein function is a long-standing problem in molecular biology with far-reaching scientific implications. Despite six decades of progress, state-of-the-art techniques cannot annotate 1/3 of microbial protein sequences, hampering our ability to exploit sequences collected from diverse organisms. In this paper, we explore an alternative methodology based on deep learning that learns the relationship between unaligned amino acid sequences and their functional annotations across all 17929 families of the Pfam database. Using the Pfam seed sequences we establish rigorous benchmark assessments that use both random and clustered data splits to control for potentially confounding sequence similarities between train and test sequences. Using Pfam full, we report convolutional networks that are significantly more accurate and computationally efficient than BLASTp, while learning sequence features such as structural disorder and transmembrane helices. Our model co-locates sequences from unseen families in embedding space, allowing sequences from novel families to be accurately annotated. These results suggest deep learning models will be a core component of future protein function prediction tools.Predicting the function of a protein from its raw amino acid sequence is a critical step for understanding the relationship between genotype and phenotype. As the cost of DNA sequencing drops and metagenomic sequencing projects flourish, fast and efficient tools that annotate open reading frames with function will play a central role in exploiting this data [1,2]. Doing so will help identify proteins that catalyze novel reactions, design new proteins that bind specific microbial targets, or build molecules that accelerate advances in biotechnology. Current practice for functional prediction of a novel protein sequence involves alignment across a large database of annotated sequences using algorithms such as 1

Constructing benchmark test sets for biological sequence analysis using independent set algorithms

“…The larger the family, the more likely it is to contain sequences that bridge together otherwise dissimilar clusters, so the procedure fails more often on alignments with many sequences. This is a concern because we and others are exploring increasingly complex and parameter-rich models for remote sequence homology recognition that can require thousands of sequences for training [8][9][10][11][12][13]. A phylogenetic approach that attempts to identify an out-group would face this same "bridge" issue.…”

Fixed-Length Protein Embeddings using Contextual Lenses

“…Using the train-test split in [4] for the sequences belonging to the 10000 lens families, we train a lens architecture to directly perform family classification for these families. We measure the accuracy of this trained lens architecture on the test subset of the lens families, which we refer to as Lens Accuracy.…”

“…We thus take the lens architecture, trained using lens training, and use it to compute fixed-length embeddings for the sequences belonging to the evaluation families. Again using the train-test split in [4], but this time for the sequences belonging to the 1000 evaluation families, we fit a 1-NN family classifier using the sequence embeddings. We fit using at most n samples per family from the train subset of the evaluation families and consider n ∈ {1, 5, 10, 50} to see how well our embeddings perform with limited data.…”

“…Bileschi et al [4] develops ProtCNN, a deep, residual, dilated CNN trained for an analogous Pfam family classification task. This model, when trained on ∼1.1M samples corresponding to 17129 families using a random train-test split, achieves an error rate of 0.495% (accuracy of 99.505%).…”

“…The encoders we use are either CNN models or frozen transformer (BERT) models. For the CNN models we baseline a simple 2-layer CNN with kernel sizes 5 and feature sizes 1024 and also compare to the more complicated ProtCNN model [4], which is a deep, residual, dilated CNN. By frozen transformer we mean a transformer model that has frozen weights.…”