A New Method to Compare the Interpretability of Rule-based Algorithms

Abstract

Interpretability is becoming increasingly important in predictive model analysis. Unfortunately, as mentioned by many authors, there is still no consensus on that idea. The aim of this article is to propose a rigorous mathematical definition of the concept of interpretability, allowing fair comparisons between any rule-based algorithms. This definition is built from three notions, each of which being quantitatively measured by a simple formula: predictivity, stability and simplicity. While predictivity has been widely studied to measure the accuracy of predictive algorithms, stability is based on the Dice-Sorensen index to compare two sets of rules generated by an algorithm using two independent samples. Simplicity is based on the sum of the length of the rules deriving from the generated model. The final objective measure of the interpretability of any rulebased algorithm ends up as a weighted sum of the three aforementioned concepts. This paper concludes with the comparison of the interpretability between four rule-based algorithms.

Framework for Evaluating Faithfulness of Local Explanations

“…In this paper, we measure faithfulness to the model. Earlier work has looked at global measures of this type [48] and measures that are specialized to neural networks [32], feature importance [4,9,43,45], rule-based explanations [24], surrogate explanation [35], or highlighted text [10,46,49].…”