Choose Your Neuron: Incorporating Domain Knowledge Through Neuron-Importance

Selvaraju, Ramprasaath R.; Chattopadhyay, Prithvijit; Elhoseiny, Mohamed; Sharma, Tilak Raj; Batra, Dhruv; Parikh, Devi; Lee, Stefan

doi:10.1007/978-3-030-01261-8_32

Cited by 38 publications

(19 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The results of NIWT [23], RN [20], and DEM [17] are obtained from the released codes on the authors' GitHub page. The rest of these results are cited directly from their published papers.…”

Section: Comparative Results Of Gzslmentioning

confidence: 99%

“…ZSKL [22] applies the well-established kernel methods to learn nonlinear mapping between the feature and attribute spaces, which contrast the existing ones that learn a linear mapping function. NIWT [23] use training instances and corresponding semantic information to learn a mapping function between the class-specific semantic and the importance of individual neurons within a deep network. The learned mapping function can predict neuron importance from knowledge regarding unseen classes and then optimize classification weights.…”

Section: A Mapping-based Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Generalized Zero Shot Learning via Synthesis Pseudo Features

Yang

et al. 2019

IEEE Access

View full text Add to dashboard Cite

Compared with conventional zero-shot learning (ZSL), generalized ZSL (GZSL) is more challenging because the test instances may come from seen and unseen classes. The most existing GZSL methods learn a visual-semantic mapping function to bridge the knowledge transfer from seen to unseen classes by using semantic information and other labeled training data. However, these methods often suffer from severe performance degradation because they ignore similar structures between different classes.To solve these problems, we propose a GZSL method that transforms GZSL problems to conventional supervised learning ones by synthesizing pseudo features for unseen classes. This technique has two key aspects. The first one is the synthesis strategy; the proposed strategy directly synthesizes the pseudo features of unseen classes contrary to current synthesis-based methods, which synthesize pseudo instances. Our method regards the combination of N features of instances as the pseudo features. These N features belong to N different classes that are similar to unseen ones. This synthesis strategy is in line with the cognitive style of human beings. The second key aspect is that we preserve the similar structures between seen and unseen classes. Inspired by the center loss method, we assign each semantic vector as the center of deep features in the training stage. This way preserves the similar structures between the classes. Such preservation can be beneficial for improving classification accuracy. The experimental results on four benchmark datasets demonstrate that our model outperforms state-of-the-art methods for the GZSL. The source code is available at https://github.com/guizilaile23/SPF-GZSL.INDEX TERMS Generalized zero-shot learning, pseudo feature synthesis, image classification, supervised learning, multi-class classification.

show abstract

“…The results of NIWT [23], RN [20], and DEM [17] are obtained from the released codes on the authors' GitHub page. The rest of these results are cited directly from their published papers.…”

Section: Comparative Results Of Gzslmentioning

confidence: 99%

Section: A Mapping-based Methodsmentioning

confidence: 99%

Generalized Zero Shot Learning via Synthesis Pseudo Features

Yang

et al. 2019

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Producing coarse localisation maps in deep neural networks that highlight input image regions essential for a task was also tackled in [ 21 ]. The authors based their research on [ 22 ], in which a method for mapping unseen objects into the dictionary of known concepts to learn classifiers for novel classes was presented. Based on this, the Human Importance-aware Network was presented in [ 23 ], where a deep neural network was encouraged to be sensitive to the same input regions as humans, which was effectively visualised.…”

Section: Related Workmentioning

confidence: 99%

Tell Me, What Do You See?—Interpretable Classification of Wiring Harness Branches with Deep Neural Networks

Kicki

Bednarek

Lembicz³

et al. 2021

Sensors

View full text Add to dashboard Cite

In the context of the robotisation of industrial operations related to manipulating deformable linear objects, there is a need for sophisticated machine vision systems, which could classify the wiring harness branches and provide information on where to put them in the assembly process. However, industrial applications require the interpretability of the machine learning system predictions, as the user wants to know the underlying reason for the decision made by the system. We propose several different neural network architectures that are tested on our novel dataset to address this issue. We conducted various experiments to assess the influence of modality, data fusion type, and the impact of data augmentation and pretraining. The outcome of the network is evaluated in terms of the performance and is also equipped with saliency maps, which allow the user to gain in-depth insight into the classifier’s operation, including a way of explaining the responses of the deep neural network and making system predictions interpretable by humans.

show abstract

“…Each prototype is then exploited to generate logical rules to provide natural language explanations. From a different perspective, Selvaraju et al propose a method to learn a map between neurons weight to semantic domain knowledge [45]. In a work more focused on unsupervised learning, Batet et al [46] exploit WordNet [47] and its taxonomic knowledge to compute semantic similarities that conduct to more interpretable clusters.…”

Section: Explanations For Non-insiders: Three Research Challenges With Symbolic Systemsmentioning

confidence: 99%

On the Integration of Knowledge Graphs into Deep Learning Models for a More Comprehensible AI—Three Challenges for Future Research

Futia

Vetrò

2020

Information

View full text Add to dashboard Cite

Deep learning models contributed to reaching unprecedented results in prediction and classification tasks of Artificial Intelligence (AI) systems. However, alongside this notable progress, they do not provide human-understandable insights on how a specific result was achieved. In contexts where the impact of AI on human life is relevant (e.g., recruitment tools, medical diagnoses, etc.), explainability is not only a desirable property, but it is -or, in some cases, it will be soon-a legal requirement. Most of the available approaches to implement eXplainable Artificial Intelligence (XAI) focus on technical solutions usable only by experts able to manipulate the recursive mathematical functions in deep learning algorithms. A complementary approach is represented by symbolic AI, where symbols are elements of a lingua franca between humans and deep learning. In this context, Knowledge Graphs (KGs) and their underlying semantic technologies are the modern implementation of symbolic AI—while being less flexible and robust to noise compared to deep learning models, KGs are natively developed to be explainable. In this paper, we review the main XAI approaches existing in the literature, underlying their strengths and limitations, and we propose neural-symbolic integration as a cornerstone to design an AI which is closer to non-insiders comprehension. Within such a general direction, we identify three specific challenges for future research—knowledge matching, cross-disciplinary explanations and interactive explanations.

show abstract

Choose Your Neuron: Incorporating Domain Knowledge Through Neuron-Importance

Cited by 38 publications

References 27 publications

Generalized Zero Shot Learning via Synthesis Pseudo Features

Generalized Zero Shot Learning via Synthesis Pseudo Features

Tell Me, What Do You See?—Interpretable Classification of Wiring Harness Branches with Deep Neural Networks

On the Integration of Knowledge Graphs into Deep Learning Models for a More Comprehensible AI—Three Challenges for Future Research

Contact Info

Product

Resources

About