Ensembling multiple raw coevolutionary features with deep residual neural networks for contact‐map prediction in CASP13

Abstract

We report the results of residue‐residue contact prediction of a new pipeline built purely on the learning of coevolutionary features in the CASP13 experiment. For a query sequence, the pipeline starts with the collection of multiple sequence alignments (MSAs) from multiple genome and metagenome sequence databases using two complementary Hidden Markov Model (HMM)‐based searching tools. Three profile matrices, built on covariance, precision, and pseudolikelihood maximization respectively, are then created from the MSAs, which are used as the input features of a deep residual convolutional neural network architecture for contact‐map training and prediction. Two ensembling strategies have been proposed to integrate the matrix features through end‐to‐end training and stacking, resulting in two complementary programs called TripletRes and ResTriplet, respectively. For the 31 free‐modeling domains that do not have homologous templates in the PDB, TripletRes and ResTriplet generated comparable results with an average accuracy of 0.640 and 0.646, respectively, for the top L/5 long‐range predictions, where 71% and 74% of the cases have an accuracy above 0.5. Detailed data analyses showed that the strength of the pipeline is due to the sensitive MSA construction and the advanced strategies for coevolutionary feature ensembling. Domain splitting was also found to help enhance the contact prediction performance. Nevertheless, contact models for tail regions, which often involve a high number of alignment gaps, and for targets with few homologous sequences are still suboptimal. Development of new approaches where the model is specifically trained on these regions and targets might help address these problems.

Protein Structure and Sequence Reanalysis of 2019-nCoV Genome Refutes Snakes as Its Intermediate Host and the Unique Similarity between Its Spike Protein Insertions and HIV-1

“…C-I-TASSER improves our previously developed I-TASSER structure prediction protocol 26 by incorporating a deep-learning-based contact map prediction. 17,27 On all 121 CASP13 targets, the average TM score of the C-I-TASSER first model (0.674) is 8.0% higher than that of I-TASSER (0.624) and 0.15% higher than that of C-QUARK (0.673), which is our only other automated CASP13 server and was ranked in second place in CASP13.…”

Recent developments in deep learning applied to protein structure prediction

“…The downside would be that the dilated filter will only be able to sample nine out of the 25 pixels and so will have “gaps.” However, these gaps can be filled by later dilated layers, so a network built with a mixture of dilated filters can cover an arbitrarily large receptive field without requiring an exponentially growing number of learnable parameters. In CASP13, dilated convolutions were used in a number of the top‐performing CNN models …”

“…In CASP13, dilated convolutions were used in a number of the top-performing CNN models. 7,41,42 Typical CNN models (eg, for image classification) take the output of one or more convolutional layers and usually downscale them with a "max pooling" operation. Max pooling simply looks for the maximum F I G U R E 1 A 2D convolutional filter (orange) is applied to an input layer (blue) to obtain the values for an output layer (green).…”

“…These so-called residual NNs (ResNets) are becoming the standard architecture for training very deep CNNs, and have been used in the best-performing methods for contact and distance prediction in CASP, 9,10,41,42,47,48 including those by the Zhang and A7D groups in CASP13.…”

A fully open-source framework for deep learning protein real-valued distances

“…It is exciting that information culled from sequences whose structures are not solved can serve as the primary input feature to predict contacts and distances. The most successful methods in the recent CASP13 experiment (http://predictioncenter.org/), a biannual protein structure and contact prediction competition, exploit such sequence databases and have unanimously demonstrated that a key to push the current progress is accurate contact and distance map prediction [2,3,4]. The distance prediction methods, in particular, are a major advancement in the area of ab initio or free modelling.…”