XAI—An Approach for Understanding Decisions Made by Neural Network

Pawade, Dipti; Dalvi, Ashwini; Gopani, Jash; Kachaliya, Chetan; Hitansh, Shah,; Shah, Hitanshu

doi:10.1007/978-981-16-0167-5_17

“…For a broader review of XAI methods, we refer the reader to [7], [23], [24], [25], [26]. Apart from the intrinsic explainable models, such as linear regression and the family of decision trees [27], the main variable importance XAI methods are: 1) Input permutation [12], [17]. The technique involves shuffling the values of one input feature and observing the effect on the model's prediction.…”

Section: State Of the Artmentioning

confidence: 99%

“…Some techniques are already available to estimate variable importance of NN models. The most commonly used are input permutation [12], which consists in perturbing the input data and observing the effect on the model's output and SHAP (SHapley Additive exPlanations) [13], which assigns an importance value to each feature by averaging over all possible coalition of features. These techniques have notable advantages: they do not depend of the topology of the ML model being analyzed (model-agnostic method) and provide quantifiable information of variable importance [14], [15].…”

Section: Introductionmentioning

confidence: 99%

Metric Tools for Sensitivity Analysis with Applications to Neural Networks

Pizarroso-Gonzalo¹,

Alfaya²,

Portela³

et al. 2023

Preprint

0

View full text Add to dashboard Cite

In this paper, a theoretical framework is proposed to study sensitivities of Machine Learning models using metric techniques. From this metric interpretation, a complete family of new quantitative metrics called α-curves is extracted. These α-curves provide information with greater depth on the importance of the input variables for a machine learning model than existing XAI methods in the literature. We demonstrate the effectiveness of the α-curves using synthetic and real datasets, comparing the results against other XAI methods for variable importance and validating the analysis results with the ground truth or literature information.

show abstract

“…For a broader review of XAI methods, we refer the reader to [7], [23], [24], [25], [26]. Apart from the intrinsic explainable models, such as linear regression and the family of decision trees [27], the main variable importance XAI methods are: 1) Input permutation [12], [17]. The technique involves shuffling the values of one input feature and observing the effect on the model's prediction.…”

Section: State Of the Artmentioning

confidence: 99%

“…Some techniques are already available to estimate variable importance of NN models. The most commonly used are input permutation [12], which consists in perturbing the input data and observing the effect on the model's output and SHAP (SHapley Additive exPlanations) [13], which assigns an importance value to each feature by averaging over all possible coalition of features. These techniques have notable advantages: they do not depend of the topology of the ML model being analyzed (model-agnostic method) and provide quantifiable information of variable importance [14], [15].…”

Section: Introductionmentioning

confidence: 99%

Metric Tools for Sensitivity Analysis with Applications to Neural Networks

Pizarroso-Gonzalo¹,

Alfaya²,

Portela³

et al. 2023

Preprint

0

View full text Add to dashboard Cite

In this paper, a theoretical framework is proposed to study sensitivities of Machine Learning models using metric techniques. From this metric interpretation, a complete family of new quantitative metrics called α-curves is extracted. These α-curves provide information with greater depth on the importance of the input variables for a machine learning model than existing XAI methods in the literature. We demonstrate the effectiveness of the α-curves using synthetic and real datasets, comparing the results against other XAI methods for variable importance and validating the analysis results with the ground truth or literature information.

show abstract

Discovering operational decisions from data—a framework supporting decision discovery from data

Leewis,

Smit,

Versendaal

2024

Decision

2

0

View full text Add to dashboard Cite

Analyzing historical decision-related data can help support actual operational decision-making processes. Decision mining can be employed for such analysis. This paper proposes the Decision Discovery Framework (DDF) designed to develop, adapt, or select a decision discovery algorithm by outlining specific guidelines for input data usage, classifier handling, and decision model representation. This framework incorporates the use of Decision Model and Notation (DMN) for enhanced comprehensibility and normalization to simplify decision tables. The framework's efficacy was tested by adapting the C4.5 algorithm to the DM45 algorithm. The proposed adaptations include (1) the utilization of a decision log, (2) ensure an unpruned decision tree, (3) the generation DMN, and (4) normalize decision table. Future research can focus on supporting on practitioners in modeling decisions, ensuring their decision-making is compliant, and suggesting improvements to the modeled decisions. Another future research direction is to explore the ability to process unstructured data as input for the discovery of decisions.

show abstract

XAI—An Approach for Understanding Decisions Made by Neural Network

Cited by 3 publications

References 11 publications

Metric Tools for Sensitivity Analysis with Applications to Neural Networks

Metric Tools for Sensitivity Analysis with Applications to Neural Networks

Metric Tools for Sensitivity Analysis with Applications to Neural Networks

Discovering operational decisions from data—a framework supporting decision discovery from data

Contact Info

Product

Resources

About