PFresGO: an attention mechanism-based deep-learning approach for protein annotation by integrating gene ontology inter-relationships

Abstract

Motivation The rapid accumulation of high-throughput sequence data demands the development of effective and efficient data-driven computational methods to functionally annotate proteins. However, most current approaches used for functional annotation simply focus on the use of protein-level information but ignore inter-relationships among annotations. Results Here, we established PFresGO, an attention-based deep-learning approach that incorporates hierarchical structures in Gene Ontology (GO) graphs and advances in natural language processing algorithms for the functional annotation of proteins. PFresGO employs a self-attention operation to capture the inter-relationships of GO terms, updates its embedding accordingly, and uses a cross-attention operation to project protein representations and GO embedding into a common latent space to identify global protein sequence patterns and local functional residues. We demonstrate that PFresGO consistently achieves superior performance across GO categories when compared with “state-of-the-art” methods. Importantly, we show that PFresGO can identify functionally important residues in protein sequences by assessing the distribution of attention weightings. PFresGO should serve as an effective tool for the accurate functional annotation of proteins and functional domains within proteins. Availability PFresGO is available for academic purposes at https://github.com/BioColLab/PFresGO. Supplementary information Supplementary data are available at Bioinformatics online.

Protein function prediction as approximate semantic entailment

“…Ontologies are not simply collections of classes; rather, ontologies are formal theories that specify some aspects of the intended meaning of a class using a logic-based language 23 . The background knowledge that is contained in the axioms of GO can be used by some machine learning models to improve predictions through knowledge-enhanced machine learning 11,12,14,15 . By incorporating the formal axioms into machine learning models, it becomes possible to leverage prior knowledge during the learning or prediction process, put constraints on the parameter search space that can improve the accuracy and efficiency of the learning process and, ultimately, make better predictions 24,25 .…”

“…These annotations are generally propagated to homologue proteins. As a result, the UniProtKB/Swiss-Prot database 3 contains manually curated GO annotations for thousands of organisms and more than 550,000 proteins.Recent protein function prediction methods rely on different sources of information such as sequence, interactions, protein tertiary structure, literature, coexpression, phylogenetic analysis or the information provided in GO [4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20] . The methods may use sequence domain annotations 5,6,8,11,21 , directly apply deep convolutional neural networks (CNN) 13 or language models such as long short-term memory neural networks 9 and transformers 14 , or use pretrained protein language models 10,15 to represent amino acid sequences.…”

DeepGO-SE: Protein function prediction as Approximate Semantic Entailment

“…Recent protein function prediction methods rely on different sources of information such as sequence, interactions, protein tertiary structure, literature, coexpression, phylogenetic analysis, or the information provided in GO [4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20]. The methods may use sequence domain annotations [5,6,8,11,21], directly apply deep convolutional neural networks (CNN) [13] or language models such as LSTMs [9] and transformers [14], or use pretrained protein language models [10,15] to represent amino acid sequences. Models may also incorporate protein-protein interactions through knowledge graph embeddings [12,16], approaches using k-nearest neighbors [21], and graph convolutional neural networks [6].…”

“…Ontologies are not simply collections of classes; rather, ontologies are formal theories that specify some aspects of the intended meaning of a class using a logic-based language [23]. The background knowledge that is contained in the axioms of GO can be used by some machine learning models to improve predictions through knowledge-enhanced machine learning [11, 12, 14, 15]. By incorporating the formal axioms into machine learning models, it becomes possible to leverage prior knowledge during the learning or prediction process, and to put constraints on the parameter search space that can improve the accuracy and efficiency of the learning process, and, ultimately, make better predictions [24, 25].…”

DeepGO-SE: Protein function prediction as Approximate Semantic Entailment

“…Recent protein function prediction methods rely on different sources of information such as sequence, interactions, protein tertiary structure, literature, coexpression, phylogenetic analysis, or the information provided in GO [4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20]. The methods may use sequence domain annotations [5,6,8,11,21], directly apply deep convolutional neural networks (CNN) [13] or language models such as LSTMs [9] and transformers [14], or use pretrained protein language models [10,15] to represent amino acid sequences. Models may also incorporate protein-protein interactions through knowledge graph embeddings [12,16], approaches using k-nearest neighbors [21], and graph convolutional neural networks [6].…”