Deep Generative Models of Protein Structure Uncover Distant Relationships Across a Continuous Fold Space

Abstract

Unresolved questions about the discrete/continuous dichotomy of protein fold space permeate structural and evolutionary biology. From protein structure comparison and classification to evolutionary analyses and function prediction, our views of fold space implicitly rest upon many assumptions that impact how we analyze, interpret and come to understand biological systems. Discrete views of fold space categorize similar folds into separate groups; unfortunately, such a ‘binning’ process inherently fails to capture many remote relationships. While hierarchical databases such as CATH, SCOP, and ECOD represent major steps forward in protein classification, we believe that a scalable, objective and conceptually flexible method that is less reliant upon assumptions and heuristics could enable a more systematic and thorough exploration of fold space and evolutionary-distant relationships. Here, we develop a structure-guided, comparative analysis of proteins, leveraging embeddings derived from deep generative models, which represent a highly-compressed, lower-dimensional space of a given protein and its sequence, structure and biophysical properties. Building upon a recent ‘Urfold’ model of protein structure, the deep generative approach developed here, termed ‘DeepUrfold’, suggests a new, mostly-continuous view of fold space—a view that extends beyond simple 3D structural/geometric similarity, towards the realm of integrated sequence↔structure↔function properties. We find that such an approach can quantitatively represent and detect evolutionarily-remote relationships that are not captured by existing methods.

Explainable Deep Generative Models, Ancestral Fragments, and Murky Regions of the Protein Structure Universe

“…We first investigated the SH3-specific (CATH 2.30.30.100), DeepUrfold-derived VAE model. This model was trained using all energy-minimized domain structures from the SH3 superfamily, along with hand-crafted biophysical features, as described in [17]. We first attempted to subject representative SH3 domains through the SH3 model and calculated relevance scores during backpropagation.…”

“…In a recent paper that introduced a DeepUrfold framework, the authors developed: (i) a preprocessed dataset, based on CATH superfamilies, that includes biophysical properties for each atom along with energy-minimized domain structures; and (ii) superfamily-specific sparse 3D-CNN VAEs [17]. Energy-minimized domain structures from a single superfamily were voxelized using a k D-tree to map atoms into 1Å 3 voxels in an overall 264 3 Å 3 discretized volume; 3D structural models were rotated randomly by sampling the SO (3) group to train a VAE model, modified from [19, 20], yielding superfamily-specific models.…”

“…For a given structure, residues containing any atom lying in the ≥ 75 th percentile were extracted to create a set of 53,480 (dis-)continuous fragments. For each community identified with Stochastic Block Modelling of the bipartite graph formed from the all superf amilies × all domains approach [17], we used the foldseek code [25] to cluster all LRP structures from domains present in each community that were processed through all superfamilies represented by the community (using TM-Align for global alignment). We select the LRP structure cluster representative from the most populated cluster in each community, resulting in a final list of top-20 "potential urfolds".…”

“…All of those findings were consistent with the prediction that the SH3 and OB comprise a distinct urfold (in this case, the SBB). This paper explores-and seeks to begin explaining-the models from [17] in more depth, by applying an approach known as layer-wise relevance propagation (LRP). In principle, explainable AI techniques such as LRP can be used to understand which atoms in the input structure are 'important', based on their spatial locations and biophysical properties (and, really, any other sorts of features that one encodes in the model).…”

“…In a recent study that developed a deep generative approach to protein structural relationships, using the Urfold model of protein structure in a framework called DeepUrfold , 20 superfamily-specific, sparse 3D-CNN variational autoencoders (VAEs) were trained for 20 different, highly-populated CATH superfamilies [17]. These DeepUrfold-trained models were shown to be agnostic to topology, as architecturally-similar SH3/OB proteins with artificially-constructed loop permutations yielded similar evidence lower bound–based (ELBO) scores; most significantly, applying community-detection methods (as stochastic block models) to the patterns of ELBO similarities led to the SH3 and OB domains clustering into similar groupings (with some intermixing).…”