Interpretable Machine Learning with an Ensemble of Gradient Boosting Machines

Abstract

A method for the local and global interpretation of a black-box model on the basis of the well-known generalized additive models is proposed. It can be viewed as an extension or a modification of the algorithm using the neural additive model. The method is based on using an ensemble of gradient boosting machines (GBMs) such that each GBM is learned on a single feature and produces a shape function of the feature. The ensemble is composed as a weighted sum of separate GBMs resulting a weighted sum of shape functions which form the generalized additive model. GBMs are built in parallel using randomized decision trees of depth 1, which provide a very simple architecture. Weights of GBMs as well as features are computed in each iteration of boosting by using the Lasso method and then updated by means of a specific smoothing procedure. In contrast to the neural additive model, the method provides weights of features in the explicit form, and it is simply trained. A lot of numerical experiments with an algorithm implementing the proposed method on synthetic and real datasets demonstrate its efficiency and properties for local and global interpretation.

Ensembles of Random SHAPs

“…The impact of every feature on the prediction in these models is determined by its corresponding shape function obtained by each neural network. Following the ideas behind these interpretation models, [35] proposed a similar model. In contrast to the method proposed by [34], an ensemble of gradient boosting machines was used in [35] instead of neural networks in order to simplify the explanation model training process.…”

“…Following the ideas behind these interpretation models, [35] proposed a similar model. In contrast to the method proposed by [34], an ensemble of gradient boosting machines was used in [35] instead of neural networks in order to simplify the explanation model training process.…”

Ensembles of Random SHAPs

“…The impact of every feature on the prediction in these models is determined by its corresponding shape function obtained by each neural network. Following ideas behind these interpretation models, Konstantinov and Utkin [28] proposed a similar model. In contrast to the method proposed by Agarwal et al [3], an ensemble of gradient boosting machines is used in [28] instead of neural networks in order to simplify the explanation model training process.…”

An Imprecise SHAP as a Tool for Explaining the Class Probability Distributions under Limited Training Data

“…The impact of every feature on the prediction in these models is determined by its corresponding shape function obtained by each neural network. Following ideas behind these interpretation models, Konstantinov and Utkin [37] proposed a similar model, but an ensemble of gradient boosting machine is used instead of neural networks in order to simplify the explanation model training process.…”

“…It should be noted that problems (37) do not have solutions for some k when inequality π U k < α L k is valid. This follows from constraints α L k ≤ α k in (39) and α k ≤ π U k in (42).…”