Fooling SHAP with Stealthily Biased Sampling

Laberge, Gabriel; Aïvodji, Ulrich; Hara, Satoshi; Marchand., Mario; Khomh, Foutse

doi:10.48550/arxiv.2205.15419

Cited by 3 publications

(2 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, black boxes can effortlessly attain high performance but their decision mechanisms are opaque and hard to understand by both experts and non-experts. Also, post-hoc explanations of these complex models have been shown to be unreliable and highly manipulable by ill-intentioned entities (Aïvodji et al, 2019;Slack et al, 2020;Dimanov et al, 2020;Laberge et al, 2022;Aïvodji et al, 2021). This conundrum between black-box or transparent designs is colloquially referred to as the "accuracy-transparency trade-off", that is, one has to choose between transparent models with lower performance or opaque models that perform well but whose explanations are not trustworthy.…”

Section: Introductionmentioning

confidence: 99%

Learning Hybrid Interpretable Models: Theory, Taxonomy, and Methods

Ferry¹,

Laberge²,

Aïvodji³

2023

Preprint

View full text Add to dashboard Cite

A hybrid model involves the cooperation of an interpretable model and a complex black box. At inference, any input of the hybrid model is assigned to either its interpretable or complex component based on a gating mechanism. The advantages of such models over classical ones are two-fold: 1) They grant users precise control over the level of transparency of the system and 2) They can potentially perform better than a standalone black box since redirecting some of the inputs to an interpretable model implicitly acts as regularization.Still, despite their high potential, hybrid models remain under-studied in the interpretability/explainability literature. In this paper, we remedy this fact by presenting a thorough investigation of such models from three perspectives: Theory, Taxonomy, and Methods. First, we explore the theory behind the generalization of hybrid models from the Probably-Approximately-Correct (PAC) perspective. A consequence of our PAC guarantee is the existence of a sweet spot for the optimal transparency of the system. When such a sweet spot is attained, a hybrid model can potentially perform better than a standalone black box. Secondly, we provide a general taxonomy for the different ways of training hybrid models: the Post-Black-Box and Pre-Black-Box paradigms. These approaches differ in the order in which the interpretable and complex components are trained. We show where the state-of-the-art hybrid models Hybrid-Rule-Set and Companion-Rule-List fall in this taxonomy. Thirdly, we implement the two paradigms in a single method: HybridCORELS, which extends the CORELS algorithm to hybrid modeling. By leveraging CORELS, Hy-bridCORELS provides a certificate of optimality of its interpretable component and precise control over transparency. We finally show empirically that HybridCORELS is competitive with existing hybrid models, and performs just as well as a standalone black box (or even better) while being partly transparent.

show abstract

Section: Introductionmentioning

confidence: 99%

Learning Hybrid Interpretable Models: Theory, Taxonomy, and Methods

Ferry¹,

Laberge²,

Aïvodji³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…To settle the above shortcomings, we adopt machine learning methods and design deep learning models to predict anesthesia recovery time. We also adopt a machine learning interpretation toolkit, such as the SHAP toolkit [22,23], to facilitate anesthesiologists to judge the importance of features. Our main contributions to this work are summarized in the following.…”

Section: Introductionmentioning

confidence: 99%

Quantitative Analysis of Anesthesia Recovery Time by Machine Learning Prediction Models

Yang

Lin

et al. 2022

Mathematics

View full text Add to dashboard Cite

It is significant for anesthesiologists to have a precise grasp of the recovery time of the patient after anesthesia. Accurate prediction of anesthesia recovery time can support anesthesiologist decision-making during surgery to help reduce the risk of surgery in patients. However, effective models are not proposed to solve this problem for anesthesiologists. In this paper, we seek to find effective forecasting methods. First, we collect 1824 patient anesthesia data from the eye center and then performed data preprocessing. We extracted 85 variables to predict recovery time from anesthesia. Second, we extract anesthesia information between variables for prediction using machine learning methods, including Bayesian ridge, lightGBM, random forest, support vector regression, and extreme gradient boosting. We also design simple deep learning models as prediction models, including linear residual neural networks and jumping knowledge linear neural networks. Lastly, we perform a comparative experiment of the above methods on the dataset. The experiment demonstrates that the machine learning method performs better than the deep learning model mentioned above on a small number of samples. We find random forest and XGBoost are more efficient than other methods to extract information between variables on postoperative anesthesia recovery time.

show abstract