NetGO 3.0: Protein Language Model Improves Large-scale Functional Annotations

Wang, Shaojun; You, Ronghui; Liu, Yunjia; Xiong, Yi; Zhu, Shanfeng

doi:10.1101/2022.12.05.519073

Cited by 10 publications

(17 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, we do not include methods that are primarily based on sequence similarity, such as BLAST, Diamond, or their combinations, as baselines. For the time-based dataset evaluation, we selected three state-of-the-art methods developed by other groups: [17], SPROF [18] and NetGO3 [19].…”

Section: Methodsmentioning

confidence: 99%

DeepGOMeta: Predicting functions for microbes

Tawfiq,

Niu,

Hoehndorf

et al. 2024

Preprint

View full text Add to dashboard Cite

Analyzing microbial samples remains computationally challenging due to their diversity and complexity. The lack of robustde novoprotein function prediction methods exacerbates the difficulty in deriving functional insights from these samples. Traditional prediction methods, dependent on homology and sequence similarity, often fail to predict functions for novel proteins and proteins without known homologs. Moreover, most of these methods have been trained on largely eukaryotic data, and have not been evaluated or applied to microbial datasets. This research introduces DeepGOMeta, a deep learning model designed for protein function prediction, as Gene Ontology (GO) terms, trained on a dataset relevant to microbes. The model is validated using novel evaluation strategies and applied to diverse microbial datasets. Data and code are available athttps://github.com/bio-ontology-research-group/deepgometa

show abstract

Section: Methodsmentioning

confidence: 99%

DeepGOMeta: Predicting functions for microbes

Tawfiq,

Niu,

Hoehndorf

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…As baselines, we trained two baseline methods DeepGO-CNN (Kulmanov and Hoehndorf, 2019) and DeepGOZero (Kulmanov and Hoehndorf, 2022) and generate predictions without using any sequence similarity component such as BLAST (Altschul et al ., 1997) or Diamond (Buchfink et al ., 2014). For time-based dataset evaluation we selected three state-of-the-art methods such as TALE (Cao and Shen, 2021a), SPROF (Yuan et al ., 2023b) and NetGO3 (Wang et al ., 2023). We used baseline methods available models to generate predictions.…”

Section: Methodsmentioning

confidence: 99%

“…On the other hand, methods that learn a classifier with PU data directly (Song et al ., 2021) rely on optimization frameworks such as Majorization Minimization (Kenneth Lange and Yang, 2000) or Support Vector Machines (Cortes and Vapnik, 1995). However, in recent years, protein function prediction has been extensively addressed with emerging deep learning techniques(Kulmanov et al ., 2017; Cao and Shen, 2021a; Yuan et al, 2023a; Wang et al, 2023).…”

Section: Introductionmentioning

confidence: 99%

Predicting protein functions using positive-unlabeled ranking with ontology-based priors

Zhapa-Camacho,

Tang,

Kulmanov

et al. 2024

Preprint

View full text Add to dashboard Cite

Automated protein function prediction is a crucial and widely studied problem in bioinformatics. Computationally, protein function is a multilabel classification problem where only positive samples are defined and there is a large number of unlabeled annotations. Most existing methods rely on the assumption that the unlabeled set of protein function annotations are negatives, inducing the false negative issue, where potential positive samples are trained as negatives. We introduce a novel approach named PU-GO, wherein we address function prediction as a positive-unlabeled ranking problem. We apply empirical risk minimization, i.e., we minimize the classification risk of a classifier where class priors are obtained from the Gene Ontology hierarchical structure. We show that our approach is more robust than other state-of-the-art methods on similarity-based and time-based benchmark datasets. Data and code are available at https://github.com/bio-ontology-research-group/PU-GO.

show abstract

“…Furthermore, transformer models have revolutionized the field by enabling the development of large Protein Language Models (LPLMs), which have emerged as transformative tools in computational biology and bioinformatics [23,24]. Similar to large Protein Language Models (LLMs) of NLP trained on large corpora of words [25], similar efforts have been applied to train LPLMs using large protein databases such as BFD100, UniRef50, and UniRef100 with trillions of protein sequences.…”

Section: Related Workmentioning

confidence: 99%

Transformer-Based Deep Learning Model with Latent Space Regularization for CRISPR-Cas Protein Sequence Classification

Nammi,

Madugula,

Pujar

et al. 2024

Preprint

View full text Add to dashboard Cite

The discovery of the CRISPR-Cas system has significantly advanced genome editing, offering vast applications in medical treatments and life sciences research. Despite their immense potential, the existing CRISPR-Cas proteins still face challenges concerning size, delivery efficiency, and cleavage specificity. Addressing these challenges necessitates a deeper understanding of CRISPR-Cas proteins to enhance the design and discovery of novel Cas proteins for precision gene editing. In this study, we performed extensive deep-learning research on CRISPR-Cas proteins, aiming to develop a classification model capable of distinguishing CAS from non-CAS proteins, as well as discriminating sub-categories of CAS proteins, specifically CAS9 and CAS12. We developed two types of deep learning models: 1) a transformer encoder-based classification model, trained from scratch; and 2) a large protein language model fine-tuned on ProtBert, pre-trained on more than 200 million proteins. To boost learning efficiency for the model trained from scratch, we introduced a novel margin-based loss function to maximize inter-class separability and intra-class compactness in protein sequence embedding latent space of a transformer encoder. The experimental results show that the Fine-Tuned ProtBert-based (FTPB) classification model achieved accuracies of 99.06\%, 94.42\%, 96.80\%, 97.57\% for CAS9 vs. Non-CAS, CAS12 vs. Non-CAS, CAS9 vs. CAS12, and multi-class classification of CAS9 vs. CAS12 vs. Non-CAS, respectively. The Latent Space Regularized Max-Margin Transformer (LSRMT) model achieved classification accuracies of 99.81\%, 99.81\%, 99.06\%, 99.27\% for the same tasks, respectively. These results demonstrate the effectiveness of the proposed Max-Margin-based latent space regularization in enhancing model robustness and generalization capabilities. Remarkably, the LSRMT model, even when trained on a significantly smaller dataset, outperformed the fine-tuned state-of-the-art large protein model. The high classification accuracies achieved by the LSRMT model demonstrate its proficiency in identifying discriminative features of CAS proteins, marking a significant step towards advancing our understanding of CAS protein structures in future research endeavors.

show abstract

NetGO 3.0: Protein Language Model Improves Large-scale Functional Annotations

Cited by 10 publications

References 31 publications

DeepGOMeta: Predicting functions for microbes

DeepGOMeta: Predicting functions for microbes

Predicting protein functions using positive-unlabeled ranking with ontology-based priors

Transformer-Based Deep Learning Model with Latent Space Regularization for CRISPR-Cas Protein Sequence Classification

Contact Info

Product

Resources

About