iForest: Interpreting Random Forests via Visual Analytics

Abstract

As an ensemble model that consists of many independent decision trees, random forests generate predictions by feeding the input to internal trees and summarizing their outputs. The ensemble nature of the model helps random forests outperform any individual decision tree. However, it also leads to a poor model interpretability, which significantly hinders the model from being used in fields that require transparent and explainable predictions, such as medical diagnosis and financial fraud detection. The interpretation challenges stem from the variety and complexity of the contained decision trees. Each decision tree has its unique structure and properties, such as the features used in the tree and the feature threshold in each tree node. Thus, a data input may lead to a variety of decision paths. To understand how a final prediction is achieved, it is desired to understand and compare all decision paths in the context of all tree structures, which is a huge challenge for any users. In this paper, we propose a visual analytic system aiming at interpreting random forest models and predictions. In addition to providing users with all the tree information, we summarize the decision paths in random forests, which eventually reflects the working mechanism of the model and reduces users' mental burden of interpretation. To demonstrate the effectiveness of our system, two usage scenarios and a qualitative user study are conducted.

explAIner: A Visual Analytics Framework for Interactive and Explainable Machine Learning

“…Others focus on specific elements, such as the neuron activation [34], hidden states of a cell [48] or action patterns of reinforcement learning algorithms [79] to allow model-specific diagnosis. Finally, some systems visualize the dataflow [43] and decision paths [86] taken by the model to enable a model diagnosis during the training process. While all these approaches allow for an integrated diagnosis, they fall short of addressing the identified issues in a subsequent refinement step.…”

Scenario-Based Requirements Elicitation for User-Centric Explainable AI

“…This is an open access post-print version; the final authenticated version is available online at https://link.springer.com/chapter/10.1007/978-3-030-57321-8_18 by © IFIP International Federation for Information Processing 2020. [22,36,43,44,55,68,71,72,80,88,89,92,93,95,96,97,98,101,102,103,104,105,106,107] 24 Yes…”

Explaining Vulnerabilities to Adversarial Machine Learning through Visual Analytics

“…In terms of decision tree-based models, BaobabView [61] proposes a natural visual representation of decision tree structures where decision criterion are visualized in the tree nodes. BOOSTVis [36] and iForest [70] also focus on explaining tree ensemble models through the use of multiple coordinated views to help explain and explore decision paths. Similarly, recent visual analytics work on deep learning [24,25,30,34,44,49,55,[63][64][65]68] tackles the issue of the low interpretability of neural network structures and supports revealing the internal logic of the training and prediction processes.…”