CNNH_PSS: protein 8-class secondary structure prediction by convolutional neural network with highway

Abstract

BackgroundProtein secondary structure is the three dimensional form of local segments of proteins and its prediction is an important problem in protein tertiary structure prediction. Developing computational approaches for protein secondary structure prediction is becoming increasingly urgent.ResultsWe present a novel deep learning based model, referred to as CNNH_PSS, by using multi-scale CNN with highway. In CNNH_PSS, any two neighbor convolutional layers have a highway to deliver information from current layer to the output of the next one to keep local contexts. As lower layers extract local context while higher layers extract long-range interdependencies, the highways between neighbor layers allow CNNH_PSS to have ability to extract both local contexts and long-range interdependencies. We evaluate CNNH_PSS on two commonly used datasets: CB6133 and CB513. CNNH_PSS outperforms the multi-scale CNN without highway by at least 0.010 Q8 accuracy and also performs better than CNF, DeepCNF and SSpro8, which cannot extract long-range interdependencies, by at least 0.020 Q8 accuracy, demonstrating that both local contexts and long-range interdependencies are indeed useful for prediction. Furthermore, CNNH_PSS also performs better than GSM and DCRNN which need extra complex model to extract long-range interdependencies. It demonstrates that CNNH_PSS not only cost less computer resource, but also achieves better predicting performance.ConclusionCNNH_PSS have ability to extracts both local contexts and long-range interdependencies by combing multi-scale CNN and highway network. The evaluations on common datasets and comparisons with state-of-the-art methods indicate that CNNH_PSS is an useful and efficient tool for protein secondary structure prediction.

SAINT: Self-Attention Augmented Inception-Inside-Inception Network Improves Protein Secondary Structure Prediction

“…PSI-BLAST returns PSSM matrix of dimension L × 21, where L is the size of a protein query sequence. This feature representation is similar to what was proposed by Zhou and Troyanskaya [26], and was subsequently used by [25,[27][28][29]].…”

“…CB513 dataset is developed by Cuff and Barton [37] and comprises 513 protein sequences and 84,107 residues. It is the most widely used benchmark dataset for evaluating protein secondary structure prediction methods [25][26][27][28][29][30]38]. This dataset, preprocessed by Zhou and Troyanskaya [26], is publicly availabe at https://www.princeton.edu/~jzthree/ datasets/ICML2014/ (last accessed June, 2019).…”

“…As such, the interest of the research community has recently shifted from Q3 prediction to relatively more challenging Q8 prediction. Quite a few deep learning methods for Q8 prediction have been proposed over the last few years [19,25,26,[28][29][30]36]. To the best of our knowledge, the first notable success in Q8 prediction methods was SSpro8 [19] which was published in 2002 and achieved 63.5% Q8 accuracy on the benchmark CB513 dataset [37], 64.9% on CASP10 and 65.6% on CASP11 [25].…”

“…Afterwards many used their dataset for training various deep learning architectures and tried to improve Q8 accuracy on the CB513 dataset. Some of the notable works include deep conditional random fields (DeepCNF) [25], cascaded convolutional and recurrent neural network (DCRNN) [27], next-step conditioned deep convolutional neural network (NCCNN) [28], multi-scale CNN with highway (CNNH PSS) [29], deep inception-inside-inception (Deep3I) network named MUFOLD-SS [30], and Deep-ACLSTM [38] with an asymmetric convolutional neural networks (ACNNs) combined with bidirectional long short-term memory (BLSTM). CRRNN (Convolutional, residual, and recurrent neural network) [39] represents another class of methods that uses various physical properties (e.g., e.g., steric parameters (graph-shape index), polarizability, normalized van der Waals volume, hydrophobicity, etc.)…”

“…Usually the models that focus more on short range dependencies (local context of the amino acid residues) face difficulties in effectively capturing the long range dependencies (interactions between amino acid residues that are close in three-dimensional space, but far from each other in the primary sequence) [22,27,40]. Various deep learning based models have been leveraged to handle the long-range interactions by using recurrent or highway networks [28,29], deeper networks with convolutional blocks [30], long short-term memory (LSTM) cells [22,27], whereas the short-range interactions have been handled by convolutional blocks of smaller window size [27,28,30]. These methods circumvent some challenging issues in capturing the non-local interactions, but have limitations of their own.…”