Learning Complete Protein Representation by Deep Coupling of Sequence and Structure

Abstract

Learning effective representations is crucial for understanding proteins and their biological functions. Recent advancements in language models and graph neural networks have enabled protein models to leverage primary or tertiary structure information to learn representations. However, the lack of practical methods to deeply co-model the relationships between protein sequences and structures has led to suboptimal embeddings. In this work, we propose CoupleNet, a network that couples protein sequence and structure to obtain informative protein representations. CoupleNet incorporates multiple levels of features in proteins, including the residue identities and positions for sequences, as well as geometric representations for tertiary structures. We construct two types of graphs to model the extracted sequential features and structural geometries, achieving completeness on these graphs, respectively, and perform convolution on nodes and edges simultaneously to obtain superior embeddings. Experimental results on a range of tasks, such as protein fold classification and function prediction, demonstrate that our proposed model outperforms the state-of-the-art methods by large margins.

Opportunities and Challenges for Machine Learning-Assisted Enzyme Engineering

“…64 The structures of active sites in unlabeled protein structures could be compared to existing structures to identify new, diverse sets of proteins with given function, using models trained on sequence and structure. 65 Structures could also be physically modeled to predict their interactions with different substrates. In principle, an ML model could be trained to combine multimodal information such as spatial descriptors of protein structures with an LLM trained on information about chemical reactions.…”

“…Alternatively, many enzymes have common “modules,” or recurring residue arrangements, which perform similar reactions . The structures of active sites in unlabeled protein structures could be compared to existing structures to identify new, diverse sets of proteins with given function, using models trained on sequence and structure . Structures could also be physically modeled to predict their interactions with different substrates.…”

Multimodal Pretraining for Unsupervised Protein Representation Learning

“…Self-supervised learning was further enhanced by leveraging a pretrained protein language model to establish a connection between sequential information in the language model and structural information in the graph neural network. Furthermore, the introduction of Cou-pleNet [42] presented a network explicitly designed to seamlessly integrate protein sequence and structure, effectively generating informative representations of proteins. This network proficiently combined residue identities and positions from sequences with geometric features derived from tertiary structures.…”

A survey on deep reinforcement learning for audio-based applications

“…Tang et al [62] focus on techniques for detecting text generated by LLMs, while Chang et al [29] have examined the various ways to evaluate LLMs. Additionally, there are a number of surveys dedicated to investigating the specialised applications of Large Models in various fields such as vision [23,24,32,33], education [34][35][36][37]63], healthcare [38,39], computational biology [42,43], computer programming [64,65], law [44][45][46]66], or robotics [47,67,68] among others. On the other hand, our survey stands apart in its exclusive focus on the applications of Large AI Models in the realm of audio signal processing, and fills an existing gap in the current body of research.…”