SAINT: Self-Attention Augmented Inception-Inside-Inception Network Improves Protein Secondary Structure Prediction

Abstract

Protein structures provide basic insight into how they can interact with other proteins, their functions and biological roles in an organism. Experimental methods (e.g., X-ray crystallography, nuclear magnetic resonance spectroscopy) for predicting the secondary structure (SS) of proteins are very expensive and time consuming. Therefore, developing efficient computational approaches for predicting the secondary structure of protein is of utmost importance. Advances in developing highly accurate SS prediction methods are mostly constrained in 3-class (Q3) structure prediction. However, 8-class (Q8) resolution of secondary structure contains more useful information and is much more challenging than the Q3 prediction. We present SAINT, a highly accurate method for Q8 structure prediction, which incorporates self-attention mechanism (a concept from natural language processing) with the Deep Inception-Inside-Inception (Deep3I) network in order to effectively capture both the short-range and long-range dependencies among the amino acid residues. SAINT offers a more interpretable framework than the typical black-box deep neural network methods. We report, on an extensive evaluation study, the performance of SAINT in comparison with the existing best methods on a collection of benchmark dataset (CB513, CASP10, and CASP11). Our results suggest that self-attention mechanism improves the prediction accuracy and outperforms the existing best alternate methods. SAINT is the first of its kind and offers the best known Q8 accuracy and interpretable results. Thus, we believe SAINT represents a major step towards the accurate and reliable prediction of secondary structures of proteins. We have made SAINT freely available as open source code at https://github.com/SAINTProtein/SAINT.

Explainable Deep Relational Networks for Predicting Compound–Protein Affinities and Contacts

“…We have tuned hyperparameters for DeepAffinity variants including learning rate ({10 −3 , 10 −4 }), batch size ({64, 128} (16 for CNN-GCN because of the limit of GPU memory) and dropout rate ({0.1, 0.2}) using random 10% of training data as validation sets. When HRNN was used to model protein sequences, we have also tuned k-mer lengths and group sizes in pairs [{ (40,30), (48,25), (30,40), (25,48), (15,80), (80,15)} for Davis and { (40,25), (50,20), (25,40), (20,50)} for KIBA and PDBbind] using the validation sets.…”

“…Additionally, attention mechanisms have been used for predictions of CPI, 18 chemical stability, 19 and protein secondary structures. 20 Assessment of interpretability for all these studies was either lacking or limited to a few case studies. We note a recent work proposing post-hoc attribution-based test to determine whether a model learns binding mechanisms.…”

“…DeepAffinity has embedded joint attentions over compound–protein component pairs and uses such joint attentions to assess origins of affinities (binding sites) or specificities. Additionally, attention mechanisms have been used for predictions of CPI, chemical stability, and protein secondary structures . Assessment of interpretability for all these studies was either lacking or limited to a few case studies.…”

Explainable Deep Relational Networks for Predicting Compound-Protein Affinities and Contacts

“…Additionally, attention mechanisms have been used for predictions of CPI, 13 chemical stability 14 and protein secondary structures. 15 Assessment of interpretability for all these studies was either lacking or limited to few case studies. We note a recent work proposing post-hoc attribution-based test to determine whether a model learns binding mechanisms.…”