Saliency methods generating visual explanatory maps representing the importance of image pixels for model classification is a popular technique for explaining neural network decisions. Hierarchical dynamic masks (HDM), a novel explanatory maps generation method, is proposed in this paper to enhance the granularity and comprehensiveness of saliency maps. First, we suggest the dynamic masks (DM), which enables multiple smallsized benchmark mask vectors to roughly learn the critical information in the image through an optimization method. Then the benchmark mask vectors guide the learning of large-sized auxiliary mask vectors so that their superimposed mask can accurately learn fine-grained pixel importance information and reduce the sensitivity to adversarial perturbations. In addition, we construct the HDM by concatenating DM modules. These DM modules are used to find and fuse the regions of interest in the remaining neural network classification decisions in the mask image in a learning-based way. Since HDM forces DM to perform importance analysis in different areas, it makes the fused saliency map more comprehensive. The proposed method outperformed previous approaches significantly in terms of recognition and localization capabilities when tested on natural and medical datasets.

The interpretability of neural networks has recently received extensive attention. The previous prototype-based explainable networks involved prototype activation in both the reasoning process and interpretation process, which requires specific explainable structures for the prototype. This makes the network less accurate as it gains interpretability. To avoid this problem, we propose a new model: decoupling prototypical network (DProtoNet), which contains three modules. 1) encoder module: we propose unrestricted masks to generate expressive features and prototypes. 2) inference module: we propose a multi-image prototype learning method to update prototypes so that the network can learn generalized prototypes.3) interpretation module: we propose multiple dynamic masks (MDM) decoder to explain the network, which generates heatmaps using the consistent activation of the original image and mask image at the detection nodes of the network. It decouples the inference module and interpretation module of a prototype-based network by avoiding the use of prototype activation to explain the network's decisions, so that the accuracy and interpretability of the network can be simultaneously improved. We test on multiple public general and medical datasets. The accuracy of our method is improved compared with the previous methods, which can be improved by up to 5%. DProtoNet achieves state-of-the-art interpretability.

No abstract

Accuracy and Diversity are two essential metrizable manifestations in generating natural and semantically correct captions. Many efforts have been made to enhance one of them with another decayed due to the trade-off gap. However, compromise does not make the progress. Decayed diversity makes the captioner a repeater, and decayed accuracy makes it a fake advisor. In this work, we exploit a novel Variational Transformer framework to improve accuracy and diversity simultaneously. To ensure accuracy, we introduce the "Invisible Information Prior" along with the "Auto-selectable GMM" to instruct the encoder to learn the precise language information and object relation in different scenes. To ensure diversity, we propose the "Range-Median Reward" baseline to retain more diverse candidates with higher rewards during the RL-based training process. Experiments show that our method achieves the simultaneous promotion of accuracy (CIDEr) and diversity (self-CIDEr), up to 1.1 and 4.8 percent, compared with the baseline. Also, our method outperforms others under the newly proposed measurement of the trade-off gap, with at least 3.55 percent promotion.Preprint. Under review.