TripletProt: Deep Representation Learning of Proteins based on Siamese Networks

Nourani, Esmaeil; Asgari, Ehsaneddin; McHardy, Alice C.; Mofrad, Mohammad R. K.

doi:10.1101/2020.05.11.088237

Cited by 5 publications

(4 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Presumably, using a more biologically-motivated proxy task will yield better insights and performance on biological data. Some methods incorporate biological information such as protein-protein interactions (Nourani et al, 2020), or structured labels from SCOP (Bepler & Berger, 2019) and PDB (Gligorijevic et al, 2019); however, high-quality curation of these labels circle back to the need for expensive experiments.…”

Section: Introductionmentioning

confidence: 99%

Self-Supervised Contrastive Learning of Protein Representations By Mutual Information Maximization

Zhang

Ghassemi

et al. 2020

Preprint

View full text Add to dashboard Cite

Pretrained embedding representations of biological sequences which capture meaningful properties can alleviate many problems associated with supervised learning in biology. We apply the principle of mutual information maximization between local and global information as a self-supervised pretraining signal for protein embeddings. To do so, we divide protein sequences into fixed size fragments, and train an autoregressive model to distinguish between subsequent fragments from the same protein and fragments from random proteins. Our model, CPCProt, achieves comparable performance to state-of-the-art self-supervised models for protein sequence embeddings on various downstream tasks, but reduces the number of parameters down to 0.9% to 8.9% of benchmarked models. Further, we explore how downstream assessment protocols affect embedding evaluation, and the effect of contrastive learning hyperparameters on empirical performance. We hope that these results will inform the development of contrastive learning methods in protein biology and other modalities.

show abstract

Section: Introductionmentioning

confidence: 99%

Self-Supervised Contrastive Learning of Protein Representations By Mutual Information Maximization

Zhang

Ghassemi

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Protein Language Models. There is growing interest in developing protein language models (pLMs) at the scale of evolution due to the abundance of 1D amino acid sequences, such as the series of ESM (Rives et al, 2019;Lin et al, 2022), TAPE (Rao et al, 2019), ProtTrans (Elnaggar et al, 2021), PRoBERTa (Nambiar et al, 2020), PMLM (He et al, 2021), ProteinLM (Xiao et al, 2021), PLUS (Min et al, 2021), Adversarial MLM (McDermott et al, 2021, ProteinBERT (Brandes et al, 2022), CARP (Yang et al, 2022a) in masked language modeling (MLM) fashion, ProtGPT2 in causal language modeling fashion, and several others (Melnyk et al, 2022a;Madani et al, 2021;Unsal et al, 2022;Nourani et al, 2021;Lu et al, 2020;Sturmfels et al, 2020;Strodthoff et al, 2020). These protein language models are able to generalize across a wide range of downstream applications and can capture evolutionary information about secondary and tertiary structures from sequences alone.…”

Section: G Related Workmentioning

confidence: 99%

Structure-informed Language Models Are Protein Designers

Zheng

Deng

Xue

et al. 2023

Preprint

View full text Add to dashboard Cite

This paper demonstrates that language models are strong structure-based protein designers. We present LM-Design, a generic approach to reprogramming sequence-based protein language models (pLMs), that have learned massive sequential evolutionary knowledge from the universe of natural protein sequences, to acquire an immediate capability to design preferable protein sequences for given folds. We conduct a structural surgery on pLMs, where a lightweight structural adapter is implanted into pLMs and endows it with structural awareness. During inference, iterative refinement is performed to effectively optimize the generated protein sequences. Experiments show that our approach outperforms the state-of-the-art methods by a large margin, leading to 4% to 12% accuracy gains in sequence recovery (e.g., 55.65% and 56.63% on CATH 4.2 and 4.3 single-chain benchmarks, and >60% when designing protein complexes). We provide extensive and in-depth analyses, which verify that LM-Design can (1) indeed leverage both structural and sequential knowledge to accurately handle structurally non-deterministic regions, (2) benefit from scaling data and model size, and (3) generalize to other proteins (e.g., antibodies and de novo proteins).

show abstract

“…We propose twin-network training of deep-learning models as a potential strategy to increase AC-sensitivity. Comparatively little work has been done to investigate twin neural network architectures (also referred to as Siamese networks [9,12,41,70]) in computational drug discovery [3,6,11,18,22,24,36,53,58,61,73,82]. However, twin networks provide a natural way to tackle chemical prediction problems on compound pairs such as AC-classification.…”

Section: Future Research: Exploring Twin-network Training Schemesmentioning

confidence: 99%

Exploring QSAR Models for Activity-Cliff Prediction

Dablander¹,

Hanser²,

Lambiotte³

et al. 2023

Preprint

View full text Add to dashboard Cite

Introduction and Methodology: Pairs of similar compounds that only differ by a small structural modification but exhibit a large difference in their binding affinity for a given target are known as activity cliffs (ACs). It has been hypothesised that QSAR models struggle to predict ACs and that ACs thus form a major source of prediction error. However, a study to explore the AC-prediction power of modern QSAR methods and its relationship to general QSAR-prediction performance is lacking. We systematically construct nine distinct QSAR models by combining three molecular representation methods (extended-connectivity fingerprints, physicochemical-descriptor vectors and graph isomorphism networks) with three regression techniques (random forests, k-nearest neighbours and multilayer perceptrons); we then use each resulting model to classify pairs of similar compounds as ACs or non-ACs and to predict the activities of individual molecules in three case studies: dopamine receptor D2, factor Xa, and SARS-CoV-2 main protease. Results and Conclusions:We observe low AC-sensitivity amongst the tested models when the activities of both compounds are unknown, but a substantial increase in AC-sensitivity when the actual activity of one of the compounds is given. Graph isomorphism features are found to be competitive with or superior to classical molecular representations for AC-classification and can thus be employed as baseline AC-prediction models or simple compound-optimisation tools. For general QSAR-prediction, however, extended-connectivity fingerprints still consistently deliver the best performance. Our results provide strong support for the hypothesis that indeed QSAR methods frequently fail to predict ACs. We propose twin-network training for deep learning models as a potential future pathway to increase AC-sensitivity and thus overall QSAR performance.

show abstract

TripletProt: Deep Representation Learning of Proteins based on Siamese Networks

Cited by 5 publications

References 26 publications

Self-Supervised Contrastive Learning of Protein Representations By Mutual Information Maximization

Self-Supervised Contrastive Learning of Protein Representations By Mutual Information Maximization

Structure-informed Language Models Are Protein Designers

Exploring QSAR Models for Activity-Cliff Prediction

Contact Info

Product

Resources

About