Learning monotone nonlinear models using the Choquet integral

Abstract

The learning of predictive models that guarantee monotonicity in the input variables has received increasing attention in machine learning in recent years. By trend, the difficulty of ensuring monotonicity increases with the flexibility or, say, nonlinearity of a model. In this paper, we advocate the so-called Choquet integral as a tool for learning monotone nonlinear models. While being widely used as a flexible aggregation operator in different fields, such as multiple criteria decision making, the Choquet integral is much less known in machine learning so far. Apart from combining monotonicity and flexibility in a mathematically sound and elegant manner, the Choquet integral has additional features making it attractive from a machine learning point of view. Notably, it offers measures for quantifying the importance of individual predictor variables and the interaction between groups of variables. Analyzing the Choquet integral from a classification perspective, we provide upper and lower bounds on its VC-dimension. Moreover, as a methodological contribution, we propose a generalization of logistic regression. The basic idea of our approach, referred to as choquistic regression, is to replace the linear function of predictor variables, which is commonly used in logistic regression to model the log odds of the positive class, by the Choquet integral. First experimental results are quite promising and suggest that the combination of monotonicity and flexibility offered by the Choquet integral facilitates strong performance in practical applications.

Monotone decomposition of 2-additive Generalized Additive Independence models

“…Methods of the second paradigm are statistical. One can mention as an example the extension of Logistic Regression to utility models incorporating interaction among criteria (Fallah Tehrani et al, 2012. Here the preference information is put into the function to optimize, and the global problem to solve is often a convex problem under linear constraints, mostly monotonicity conditions.…”

Representation of maxitive measures: An overview

“…In [26] Choquet built on the same idea to create the tool now called the Choquet integral; it was revived by Schmeidler [119,120]; its theoretical properties were developed e.g. by Greco [59], Groes et al [61], König [72]; it has found numerous applications, as in statistics and data mining (see Murofushi and Sugeno [95], Grabisch [56], Wang, Leung, and Klir [132], Fallah Tehrani et al [44]), game theory and mathematical economics (see Gilboa and Schmeidler [52], Heilpern [65]), decision theory (see Chateauneuf [24], Grabisch [54,55], Grabisch and Roubens [58], Grabisch and Labreuche [57], Mayag, Grabisch, and Labreuche [86]), insurance and finance (see Chateauneuf, Kast, and Lapied [25], Castagnoli, Maccheroni, and Marinacci [20]).…”

Preference Learning Using the Choquet Integral: The Case of Multipartite Ranking

“…• As opposed to many other models used in machine learning, the Choquet integral guarantees monotonicity in all criteria [49]. This is a reasonable property of a utility function which is often required in practice.…”