LINDA-BN: An interpretable probabilistic approach for demystifying black-box predictive models

Moreira, Catarina; Chou, Yu-Liang; Velmurugan, Mythreyi; Ouyang, Chun; Sindhgatta, Renuka; Bruza, Peter

doi:10.1016/j.dss.2021.113561

Cited by 57 publications

(15 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The decision surface of the model becomes smoother as the input space is restricted. Local interpretability is often achieved through the use of local example-based techniques or local surrogates, which simulate a limited region surrounding an example [24,25,23,26].…”

Section: Model-agnostic Counterfactual Algorithmsmentioning

confidence: 99%

Benchmark Evaluation of Counterfactual Algorithms for XAI: From a White Box to a Black Box

Chou¹,

Hsieh²,

Moreira³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Counterfactual explanations have recently been brought to light as a potentially crucial response to obtaining human-understandable explanations from predictive models in Explainable Artificial Intelligence (XAI). This counterfactual explanation claims to be capable of identifying the smallest feature change capable of changing a prediction. It produces an explanation that is more causally informative than factual explanations in terms of expanding human knowledge of decision making. Despite the fact that various counterfactual algorithms have been proposed, the state of the art research still lacks on standardised protocols to evaluate the quality of counterfactual explanations.In this work, we conducted a benchmark evaluation across different model agnostic counterfactual algorithms in the literature (DiCE, WatcherCF, prototype, unjustifiedCF), and we investigated the counterfactual generation process on different types of machine learning models ranging from a white box (decision tree) to a grey-box (random forest) and a black box (neural network). We evaluated the different counterfactual algorithms using several metrics including proximity, interpretability and functionality for five datasets.The main findings of this work are the following: (1) without guaranteeing plausibility in the counterfactual generation process, one cannot have meaningful evaluation results. This means that all explainable counterfactual algorithms that do not take into consideration plausibility in their internal mechanisms cannot be evaluated with the current state of the art evaluation metrics; (2) the counterfactual generated are not impacted by the different types of machine learning models; (3) DiCE was the only tested algorithm that was able to generate actionable and plausible counterfactuals, because it provides mechanisms to constraint features; (4) WatcherCF and UnjustifiedCF are limited to continuous variables and can not deal with categorical data.

show abstract

Section: Model-agnostic Counterfactual Algorithmsmentioning

confidence: 99%

Benchmark Evaluation of Counterfactual Algorithms for XAI: From a White Box to a Black Box

Chou¹,

Hsieh²,

Moreira³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…On the other hand, GBM is computationally effective and often outperforms other algorithms [53,54]. The three are also regarded interpretable, i.e., essential for obtaining novel insights (e.g., new causal links between explanatory and response variables) and troubleshooting the models (i.e., detecting and diagnosing biases in the input data and trained models) for improved satellite-based value-added products [55] as opposed to kernel-based and deep learning algorithms such as Kernel Ridge Regression (KRR), Support Vector Machines (SVM), and Neural Networks (NN), which are "black-boxes", complex, and computationally expensive [56]. Therefore, the comparison of these three MLRAs in the context of crop biophysical parameter estimation is worthwhile to elucidate their capabilities under the same environmental and acquisition conditions and consistency to previous performances.…”

Section: Machine Learning Regression Algorithmsmentioning

confidence: 99%

Estimating Crop Biophysical Parameters Using Machine Learning Algorithms and Sentinel-2 Imagery

2021

View full text Add to dashboard Cite

Global food security is critical to eliminating hunger and malnutrition. In the changing climate, farmers in developing countries must adopt technologies and farming practices such as precision agriculture (PA). PA-based approaches enable farmers to cope with frequent and intensified droughts and heatwaves, optimising yields, increasing efficiencies, and reducing operational costs. Biophysical parameters such as Leaf Area Index (LAI), Leaf Chlorophyll Content (LCab), and Canopy Chlorophyll Content (CCC) are essential for characterising field-level spatial variability and thus are necessary for enabling variable rate application technologies, precision irrigation, and crop monitoring. Moreover, robust machine learning algorithms offer prospects for improving the estimation of biophysical parameters due to their capability to deal with non-linear data, small samples, and noisy variables. This study compared the predictive performance of sparse Partial Least Squares (sPLS), Random Forest (RF), and Gradient Boosting Machines (GBM) for estimating LAI, LCab, and CCC with Sentinel-2 imagery in Bothaville, South Africa and identified, using variable importance measures, the most influential bands for estimating crop biophysical parameters. The results showed that RF was superior in estimating all three biophysical parameters, followed by GBM which was better in estimating LAI and CCC, but not LCab, where sPLS was relatively better. Since all biophysical parameters could be achieved with RF, it can be considered a good contender for operationalisation. Overall, the findings in this study are significant for future biophysical product development using RF to reduce reliance on many algorithms for specific parameters, thus facilitating the rapid extraction of actionable information to support PA and crop monitoring activities.

show abstract

“…The aim of the second phase is to develop a method to test the fidelity of explanations for a black box, using a white box model as a substitute to determine the appropriateness of the method and parameters used in the method. Specifically, the approach we proposed in a previous work will be refined and extended to better evaluate the fidelity of black box models (Velmurugan et al, 2021). This approach is adapted from an ablation-based method used to test the internal fidelity of tabular and text data, which tests the correctness of the explanation.…”

Section: Phase 2: White Box As Proxy For Black Boxmentioning

confidence: 99%

“…While this approach of "removal" of features is relatively simple in image or text data, this approach will not hold for tabular data, were "gaps" in the input are automatically imputed by the predictive model, or are otherwise treated as some improbable value, such as infinity. Thus, in a previous work (Velmurugan et al, 2021), rather than removing features, we attempted to alter them through perturbation to reflect a value outside of the range of values considered to be relevant by the explanation.…”

Section: Phase 2: White Box As Proxy For Black Boxmentioning

confidence: 99%

See 1 more Smart Citation

Developing a Fidelity Evaluation Approach for Interpretable Machine Learning

Velmurugan¹,

Ouyang²,

Moreira³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Although modern machine learning and deep learning methods allow for complex and in-depth data analytics, the predictive models generated by these methods are often highly complex, and lack transparency. Explainable AI (XAI) methods are used to improve the interpretability of these complex models, and in doing so improve transparency. However, the inherent fitness of these explainable methods can be hard to evaluate. In particular, methods to evaluate the fidelity of the explanation to the underlying black box require further development, especially for tabular data. In this paper, we (a) propose a three phase approach to developing an evaluation method; (b) adapt an existing evaluation method primarily for image and text data to evaluate models trained on tabular data; and (c) evaluate two popular explainable methods using this evaluation method. Our evaluations suggest that the internal mechanism of the underlying predictive model, the internal mechanism of the explainable method used and model and data complexity all affect explanation fidelity. Given that explanation fidelity is so sensitive to context and tools and data used, we could not clearly identify any specific explainable method as being superior to another.

show abstract

LINDA-BN: An interpretable probabilistic approach for demystifying black-box predictive models

Cited by 57 publications

References 11 publications

Benchmark Evaluation of Counterfactual Algorithms for XAI: From a White Box to a Black Box

Benchmark Evaluation of Counterfactual Algorithms for XAI: From a White Box to a Black Box

Estimating Crop Biophysical Parameters Using Machine Learning Algorithms and Sentinel-2 Imagery

Developing a Fidelity Evaluation Approach for Interpretable Machine Learning

Contact Info

Product

Resources

About