A Visual Designer of Layer‐wise Relevance Propagation Models

Abstract

Layer‐wise Relevance Propagation (LRP) is an emerging and widely‐used method for interpreting the prediction results of convolutional neural networks (CNN). LRP developers often select and employ different relevance backpropagation rules and parameters, to compute relevance scores on input images. However, there exists no obvious solution to define a “best” LRP model. A satisfied model is highly reliant on pertinent images and designers' goals. We develop a visual model designer, named as VisLRPDesigner, to overcome the challenges in the design and use of LRP models. Various LRP rules are unified into an integrated framework with an intuitive workflow of parameter setup. VisLRPDesigner thus allows users to interactively configure and compare LRP models. It also facilitates relevance‐based visual analysis with two important functions: relevance‐based pixel flipping and neuron ablation. Several use cases illustrate the benefits of VisLRPDesigner. The usability and limitation of the visual designer is evaluated by LRP users.

The Transform-and-Perform Framework: Explainable Deep Learning Beyond Classification

“…) has also been shown for goals ranging from model design [19], [21], diagnosing transfer learning process [46], and model exploration [18], [35]. However, the level of control on the components is limited.…”

“…However, the level of control on the components is limited. For example, input analysis is often limited to instance level [18], [35] or based on predefined class groups [19], [21]. These limitations make it challenging to use VA systems on large datasets or where intra-class variation is high.…”

“…• Analysis of complex models can be supported by extracting deeper insights [20] via the base model elements . For example, previous works have utilized gradients [67], layer-wise relevance propagation [18], [68], activation maximization [69], and neuron importance [70]…”

“…Huang et al [18], Das et al [42] Given an architecture, explore the input preprocessing to apply to boost performance.…”

“…In recent years, visual analytics (VA) has shown promise in DL model interpretation [12], including model understanding [13], [17], [18], compression and design [19], [20], [21], out-of-distribution analysis [22], and hyperparameter tuning [23]. The focus has primarily been on classification [15], which does not exhibit the complexities of H-H problems.…”