ExplaiNN: interpretable and transparent neural networks for genomics

Abstract

Sequence-based deep learning models, particularly convolutional neural networks (CNNs), have shown superior performance on a wide range of genomic tasks. A key limitation of these models is the lack of interpretability, slowing their broad adoption by the genomics community. Current approaches to model interpretation do not readily reveal how a model makes predictions, can be computationally intensive, and depend on the implemented architecture. Here, we introduce ExplaiNN, an adaptation of neural additive models1 for genomic tasks wherein predictions are computed as a linear combination of multiple independent CNNs, each consisting of a single convolutional filter and fully connected layers. This approach brings together the expressivity of CNNs with the interpretability of linear models, providing global (cell state level) as well as local (individual sequence level) insights of the biological processes studied. We use ExplaiNN to predict transcription factor (TF) binding and chromatin accessibility states, demonstrating performance levels comparable to state-of-the-art methods, while providing a transparent view of the model’s predictions in a straightforward manner. Applied to de novo motif discovery, ExplaiNN detects equivalent motifs to those obtained from specialized algorithms across a range of datasets. Finally, we present ExplaiNN as a plug and play platform in which pre-trained TF binding models and annotated position weight matrices from reference databases can be combined in a simple framework. We expect that ExplaiNN will accelerate the adoption of deep learning by biological domain experts in their daily genomic sequence analyses.

A multimorphic mutation in IRF4 causes human autosomal dominant combined immunodeficiency

“…Emphasizing the neomorphic activity of IRF4 T95R , more than 35% of the peaks (versus <10% in IRF4 WT ) corresponded to "non-ChIPable" IRF4 regions [i.e., they are not reported in the ReMap database (21), which aggregates IRF4 ChIPseq data from B cells, T cells, plasmablasts, and various cell lines], and about 33% do not overlap any of the >1 million candidate cisregulatory elements from ENCODE (22) (versus ~7% in IRF4 WT ). Applying a new deep learning tool, ExplaiNN (explainable neural networks) (23), we separately identified motifs de novo in four different datasets, including the patient and healthy control ChIP-seq datasets and two custom datasets describing the binding of IRF4 to either AICE or EICE sites in GM12878 cells. Next, we used these motifs to initialize a "surrogate" ExplaiNN model in a process known as transfer learning with which to evaluate their importance toward the IRF4 T95R -specific, IRF4 WT -specific, or common component of the ChIP-seq data (fig.…”

“…Then, the enrichment of each 8-nucleotide oligomer was computed as the logarithm to the base 2, resulting from the division between the number of occurrences of that 8-nucleotide oligomer in the last and first SELEX cycles. Motifs were obtained using ExplaiNN (23) (see the "Deep learning models" section in the Supplementary Materials and Methods).…”

EUGENe: A Python toolkit for predictive analyses of regulatory sequences

“…Interpretation of trained models has been crucial for deciphering aspects of the cis-regulatory code and is a core aspect of the EUGENe workflow (interpret module, Figure 1c). There are many strategies for model interpretation in genomics [44][45][46][47][48][49][50][51] , but three categories are repeatedly used and thus implemented in EUGENe: filter visualization, feature attribution and in silico experimentation (Supplementary Figure 1). Filter visualization is applicable to model architectures that begin with a set of convolutional filters and involves using the set of sequences that significantly activate a given filter (maximally activating subsequences) to generate a position frequency matrix (PFM) (Supplementary Figure 1a).…”

“…Similarly, though any user could develop their own methods for benchmarking EUGENe models against shallow machine learning models like gkm-SVMs 74 or random forests 75 , we plan on integrating functionality for automating this process. Finally, we plan on expanding EUGENe's dataset, model, metric and interpretation [45][46][47][48][49]51 library to encompass a larger portion of those available in the field.…”

Deep Learning in Population Genetics

“…Intriguingly, novel applications may go beyond classic inferential tasks and include other aims, such as efficient data compression or generation of synthetic experimental data sets. Likewise, solutions for making neural networks a “transparent-box,” such as neural additive models ( Novakovsky et al 2022 ) and symbolic metamodeling ( Alaa and van der Schaar 2019 ), will facilitate the adoption of deep learning among empiricists.…”