Discriminating Spatial and Temporal Relevance in Deep Taylor Decompositions for Explainable Activity Recognition

Abstract

Current techniques for explainable AI have been applied with some success to image processing. The recent rise of research in video processing has called for similar work in deconstructing and explaining spatio-temporal models. While many techniques are designed for 2D convolutional models, others are inherently applicable to any input domain. One such body of work, deep Taylor decomposition, propagates relevance from the model output distributively onto its input and thus is not restricted to image processing models. However, by exploiting a simple technique that removes motion information, we show that it is not the case that this technique is effective as-is for representing relevance in non-image tasks. We instead propose a discriminative method that produces a naïve representation of both the spatial and temporal relevance of a frame as two separate objects. This new discriminative relevance model exposes relevance in the frame attributed to motion, that was previously ambiguous in the original explanation. We observe the effectiveness of this technique on a range of samples from the UCF-101 action recognition dataset, two of which are demonstrated in this paper.

SWAG-V: Explanations for Video using Superpixels Weighted by Average Gradients

“…This is exasperated as often there is no cohesion between frames leading to explanations produced this way to appear like noise. Hiley et al [11,12] have subsequently proposed a modification to these gradient-based techniques to make them more suitable to action recognition networks.…”

“…Subtracting the insertion score from 1 means that as both scores approach 0, the better they are. We compute the combined scores for w c = [18,20] and w g = [8,9,10,11,12]. These results are shown in Figure 2.…”

Temporal Relevance Analysis for Video Action Models

“…Montavon et al proposed the layer-wise relevance propagation (LRP) to show the relevance between each pixel and the predicted class [7]; moreover, contrastive LRP further considers the effects of non-predicted classes to enhance the relevance to the targeted class [20]. Several works extend LRP for spatio temporal explainable methods (e.g., [3,21,22,43]). Hiley et al [21] surveyed a few frameworks that are used to explain action recognition models based on LRP or CAM approaches.…”

“…In this paper, we provide a deep analysis of temporal modeling for action recognition. Previous works focus on performance benchmark [10,54], spatiotemporal feature visualization [18,40] or salieny analysis [5,22,39,48] to gain better understanding of action models. For example, comprehensive studies of CNNbased models have been conducted recently in [10,54] to compare performance of different action models.…”