Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Models

Apley, Daniel W.; Zhu, Jingyu

doi:10.48550/arxiv.1612.08468

Cited by 54 publications

(82 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Taking our dataset as an example, TT MOD is somewhat correlated with other travel time variables, since the origin-destination distance is the same. To overcome this issue, a potential direction is to apply the accumulated local effects (ALE) plot (Apley, 2016), by only visualizing how the model predictions change in a small "window" around a particular feature value. This idea can be applied to PDPs, ICE plots, and their generalizations proposed in the paper.…”

Section: Discussionmentioning

confidence: 99%

Modeling Heterogeneity in Mode-Switching Behavior Under a Mobility-on-Demand Transit System: An Interpretable Machine Learning Approach

Zhao,

Yan,

Van Hentenryck

2019

Preprint

View full text Add to dashboard Cite

Over the last decade, machine learning has been applied to many engineering and scientific disciplines, often achieving high predictive accuracy. Recent years, however, have witnessed an increased focus on interpretability and the use of machine learning to inform policy analysis and decision making. This paper applies machine learning to examine travel behavior and, in particular, on modeling changes in travel modes when individuals are presented with a novel (on-demand) mobility option. It addresses the following question: Can machine learning be applied to model individual taste heterogeneity (preference heterogeneity for travel modes and response heterogeneity to travel attributes) in travel mode choice?This paper first develops a high-accuracy classifier to predict mode-switching behavior under a hypothetical Mobility-on-Demand Transit system (i.e., stated-preference data), which represents the case study underlying this research. We show that this classifier naturally captures individual heterogeneity available in the data. Moreover, the paper derives insights on heterogeneous switching behaviors through the generation of marginal effects and elasticities by current travel mode, partial dependence plots, individual conditional expectation plots. The paper also proposes two new model-agnostic interpretation tools for machine learning, i.e., conditional partial dependence plots and conditional individual partial dependence plots, specifically designed to examine response heterogeneity. The results on the case study show that the machine-learning classifier, together with model-agnostic interpretation tools, provides valuable insights on travel mode switching behavior for different individuals and population segments. For example, the existing drivers are more sensitive to additional pickups than people using other travel modes, and current transit users are generally willing to share rides but reluctant to take any additional transfers. These insights can be used to inform transportation planning and the design of new mobility services.

show abstract

Section: Discussionmentioning

confidence: 99%

Modeling Heterogeneity in Mode-Switching Behavior Under a Mobility-on-Demand Transit System: An Interpretable Machine Learning Approach

Zhao,

Yan,

Van Hentenryck

2019

Preprint

View full text Add to dashboard Cite

show abstract

“…Alibi provides a set of counterfactual explanations, such as cem, and, interestingly, an implementation of anchor [103]. Regarding global explanation methods, Alibi contains ale (Accumulated Local Effects) [11], which is a method based on partial dependence plots [59]. FAT-Forensics takes into account fairness, accountability and transparency.…”

Section: Explanation Toolboxesmentioning

confidence: 99%

“…For reproducibility reasons, we fixed the random seed 11. We refer the intrested reader to: https://christophm.github.io/interpretable-ml-book/shapley.…”

mentioning

confidence: 99%

Benchmarking and Survey of Explanation Methods for Black Box Models

Bodria¹,

Giannotti²,

Guidotti³

et al. 2021

Preprint

View full text Add to dashboard Cite

The widespread adoption of black-box models in Artificial Intelligence has enhanced the need for explanation methods to reveal how these obscure models reach specific decisions. Retrieving explanations is fundamental to unveil possible biases and to resolve practical or ethical issues. Nowadays, the literature is full of methods with different explanations. We provide a categorization of explanation methods based on the type of explanation returned. We present the most recent and widely used explainers, and we show a visual comparison among explanations and a quantitative benchmarking.

show abstract

“…Motivated by (Apley, 2016), we propose the gradient tracking technique to achieve the input variable selection for a selected output variable for a particular sample for local interpretability. This is done through tracking back the gradient of the identified output variable y kj according to each individual input variable by backpropagation through the output linkage function y kj = g kj (h k ; θ g k ) and the state transition matrix…”

Section: Improve Model Global Interpretabilitymentioning

confidence: 99%

Deep multistage multi-task learning for quality prediction of multistage manufacturing systems

Yan

Sergin

Brenneman

et al. 2021

Journal of Quality Technology

View full text Add to dashboard Cite

In multistage manufacturing systems, modeling multiple quality indices based on the process sensing variables is important. However, the classic modeling technique predicts each quality variable one at a time, which fails to consider the correlation within or between stages. We propose a deep multistage multi-task learning framework to jointly predict all output sensing variables in a unified end-to-end learning framework according to the sequential system architecture in the MMS. Our numerical studies and real case study have shown that the new model has a superior performance compared to many benchmark methods as well as great interpretability through developed variable selection techniques.

show abstract

Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Models

Cited by 54 publications

References 8 publications

Modeling Heterogeneity in Mode-Switching Behavior Under a Mobility-on-Demand Transit System: An Interpretable Machine Learning Approach

Modeling Heterogeneity in Mode-Switching Behavior Under a Mobility-on-Demand Transit System: An Interpretable Machine Learning Approach

Benchmarking and Survey of Explanation Methods for Black Box Models

Deep multistage multi-task learning for quality prediction of multistage manufacturing systems

Contact Info

Product

Resources

About