Consistent Regression using Data-Dependent Coverings

Abstract

We introduce a novel method to generate interpretable regression function estimators. The idea is based on data-dependent coverings. The aim is to extract from the data a covering of the feature space instead of a partition. The estimator predicts the empirical conditional expectation over the cells of the partitions generated from the coverings. Thus, such estimator has the same form as those issued from data-dependent partitioning algorithms. We give sufficient conditions to ensure the consistency, avoiding the sufficient condition of shrinkage of the cells that appears in the former literature. Doing so, we reduce the number of covering elements. We show that such coverings are interpretable and each element of the covering is tagged as significant or insignificant.The proof of the consistency is based on a control of the error of the empirical estimation of conditional expectations which is interesting on its own.

A New Method to Compare the Interpretability of Rule-based Algorithms

“…It can be adapted to the various desiderata of the statistician by choosing appropriate the coefficients in the interpretability formula (10). An application on four rule-based algorithm: Decision Tree algorithm [6], RuleFit [12], Covering Algorithm [20] and RICE [16], shows how to use and analyse the interpretability value. This application will be extended to others well-known rule-based algorithms such as C4.5 [8], RIPPER [9], Ender [13] and SIRUS [15] in a further work.…”

“…In [20], authors have introduced a notion of interpretability score based on the sum of the length of all the rules constituting the predictive model.…”

“…The aim of this application is to compare four rule-based algorithms: the Decision Tree algorithm (DT) [6], RuleFit (RF) [12], the Covering Algorithm (CA) [20] and RICE [16]. Their parametrization is summarized in Table 1.…”