End-to-End Protein Normal Mode Frequency Predictions Using Language and Graph Models and Application to Sonification

Abstract

The prediction of mechanical and dynamical properties of proteins is an important frontier, especially given the greater availability of proteins structures. Here we report a series of models that provide end-to-end predictions of nanodynamical properties of proteins, focused on high-throughput normal mode predictions directly from the amino acid sequence. Using neural network models within the family of Natural Language Processing and graph-based methods, we offer atomistically based mechanistic predictions of key protein mechanical features. The models include an end-to-end long short-term memory (LSTM) model, an end-to-end transformer model, a graph-based transformer model, and an equivariant graph neural network. All four models show exceptional performance, with the graph-based transformer architecture offering the best results but at the cost of requiring a graph structure as input. Conversely, the LSTM and transformer models offer end-to-end sequenceto-property prediction capabilities, providing efficient avenues for protein engineering, analysis, and design. We compare our results against published data based on a Principal Neighborhood Aggregation graph neural network, revealing that the transformer model offers better performance while also being able to predict a large set of the first 64 normal mode frequencies, simultaneously. The use of the end-to-end transformer model may facilitate other downstream applications through the use of transfer learning, and it offers a comprehensive prediction of dynamical properties without any structural knowledge, directly from the amino acid sequence. We demonstrate a potential application in scientific sonification, where the normal mode frequencies are transposed to generate audible signals for a detailed analysis of subtle changes of protein sequences.

Molecular Contrastive Pretraining with Collaborative Featurizations

“…91 Our approach can also aid in predictions by integrating various protein features like amino acid sequences and protein structures, which have complementary inductive bias. 92 Moreover, the concept of integrating multiple featurizations holds potential for application in chemical reactions. In certain cases, reaction coordinates might be more discernible and easier to learn using one type of featurization over others.…”

“…For instance, in the domain of protein property prediction, modeling mechanical and dynamical properties could be very important, as exemplified in normal-mode frequency prediction . Our approach can also aid in predictions by integrating various protein features like amino acid sequences and protein structures, which have complementary inductive bias . Moreover, the concept of integrating multiple featurizations holds potential for application in chemical reactions.…”

Transformer-Based Deep Learning Model with Latent Space Regularization for CRISPR-Cas Protein Sequence Classification

“…Oliveira et al employed Transformer models for protein function annotation, showcasing their proficiency in decoding amino acid sequence patterns [18]. Hu et al developed a graph-based Transformer model to predict protein normal mode frequencies and demonstrated the superior capabilities of transformer models to predict protein properties using protein sequence data [19]. Du et al and Tang et al employed transformer models to predict human secretory proteins and identify plasmid contigs, respectively, exemplifying the versatility and potential of transformer models to advance our comprehension of protein sequences and structures across diverse contexts [20,21].…”

ForceGen: End-to-end de novo protein generation based on nonlinear mechanical unfolding responses using a language diffusion model

“…End-to-end models based on deep learning that predict various structural features [e.g., secondary structure type and content (28)(29)(30)(31)(32)(33), binding sites (34), and surfaces (35)] and properties [e.g., solubility (16,36,37), melting temperature (38), natural vibrational frequencies (39,40), and strength (41)] for given sequences have also been reported. At the sample time, the inverse design of de novo proteins that meet desired structural or property features presents a more challenging task.…”