Visualizing complex feature interactions and feature sharing in genomic deep neural networks

Abstract

Background Visualization tools for deep learning models typically focus on discovering key input features without considering how such low level features are combined in intermediate layers to make decisions. Moreover, many of these methods examine a network’s response to specific input examples that may be insufficient to reveal the complexity of model decision making. Results We present DeepResolve, an analysis framework for deep convolutional models of genome function that visualizes how input features contribute individually and combinatorially to network decisions. Unlike other methods, DeepResolve does not depend upon the analysis of a predefined set of inputs. Rather, it uses gradient ascent to stochastically explore intermediate feature maps to 1) discover important features, 2) visualize their contribution and interaction patterns, and 3) analyze feature sharing across tasks that suggests shared biological mechanism. We demonstrate the visualization of decision making using our proposed method on deep neural networks trained on both experimental and synthetic data. DeepResolve is competitive with existing visualization tools in discovering key sequence features, and identifies certain negative features and non-additive feature interactions that are not easily observed with existing tools. It also recovers similarities between poorly correlated classes which are not observed by traditional methods. DeepResolve reveals that DeepSEA’s learned decision structure is shared across genome annotations including histone marks, DNase hypersensitivity, and transcription factor binding. We identify groups of TFs that suggest known shared biological mechanism, and recover correlation between DNA hypersensitivities and TF/Chromatin marks. Conclusions DeepResolve is capable of visualizing complex feature contribution patterns and feature interactions that contribute to decision making in genomic deep convolutional networks. It also recovers feature sharing and class similarities which suggest interesting biological mechanisms. DeepResolve is compatible with existing visualization tools and provides complementary insights. Electronic supplementary material The online version of this article (10.1186/s12859-019-2957-4) contains supplementary material, which is available to authorized users.

Improving representations of genomic sequence motifs in convolutional networks with exponential activations

“…However, attribution methods explored here are first-order interpretability methods, defining the importance of individual nucleotides -not the importance of the entire motif on model predictions. Although recent progress is extending these class of methods to second-order attributions 53,[56][57][58] , they cannot uncover the effect size of motifs on model predictions. Global interpretability analysis via in silico experiments is one avenue that shows great promise in uncovering the importance of whole features 59 .…”

A Self-Attention Model for Inferring Cooperativity between Regulatory Features

“…The major drawback of DFIM is that it is computationally expensive: the interactions are inferred in a separate post-processing step and involves recalculation of network gradients. We note that the recent DeepResolve method infers feature importance and whether a feature participates in interactions with other features, but does not infer pairs of interacting features explicitly [Liu et al, 2019].…”

Global Importance Analysis: An Interpretability Method to Quantify Importance of Genomic Features in Deep Neural Networks

“…Recent progress has expanded the ability to probe interactions between putative motifs 37–39 . For instance, MaxEnt Interpretation uses Markov Chain Monte Carlo to sample sequences that produce a similar activation profile in the penultimate layer of the DNN 37 , allowing for downstream analysis of these sequences.…”