Nearest Neighbour Classification with Monotonicity Constraints

Duivesteijn, Wouter; Feelders, Ad

doi:10.1007/978-3-540-87479-9_38

Cited by 76 publications

(72 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, if models violate these constraints they may not be accepted by experts as valid, conforming to monotonicity constraints improves model acceptance [7,10].…”

Section: Monotonicitymentioning

confidence: 99%

Using an Ant Colony Optimization Algorithm for Monotonic Regression Rule Discovery

Brookhouse

Otero

2016

Proceedings of the Genetic and Evolutionary Computation Conference 2016

View full text Add to dashboard Cite

Many data mining algorithms do not make use of existing domain knowledge when constructing their models. This can lead to model rejection as users may not trust models that behave contrary to their expectations. Semantic constraints provide a way to encapsulate this knowledge which can then be used to guide the construction of models. One of the most studied semantic constraints in the literature is monotonicity, however current monotonically-aware algorithms have focused on ordinal classification problems. This paper proposes an extension to an ACO-based regression algorithm in order to extract a list of monotonic regression rules. We compared the proposed algorithm against a greedy regression rule induction algorithm that preserves monotonic constraints and the well-known M5' Rules. Our experiments using eight publicly available data sets show that the proposed algorithm successfully creates monotonic rules while maintaining predictive accuracy.

show abstract

“…However, if models violate these constraints they may not be accepted by experts as valid, conforming to monotonicity constraints improves model acceptance [7,10].…”

Section: Monotonicitymentioning

confidence: 99%

Using an Ant Colony Optimization Algorithm for Monotonic Regression Rule Discovery

Brookhouse

Otero

2016

Proceedings of the Genetic and Evolutionary Computation Conference 2016

View full text Add to dashboard Cite

show abstract

“…Meanwhile, several machine learning algorithms have been modified so as to guarantee monotonicity in attributes, including nearest neighbor classification (Duivesteijn and Feelders 2008), neural networks (Sill 1998), decision tree learning (Ben-David 1995;Potharst and Feelders 2002), rule induction (Dembczyński et al 2009), as well as methods based on isotonic separation (Chandrasekaran et al 2005) and piecewise linear models (Dembczyński et al 2006).…”

Section: Related Workmentioning

confidence: 99%

Learning monotone nonlinear models using the Choquet integral

et al. 2012

View full text Add to dashboard Cite

The learning of predictive models that guarantee monotonicity in the input variables has received increasing attention in machine learning in recent years. By trend, the difficulty of ensuring monotonicity increases with the flexibility or, say, nonlinearity of a model. In this paper, we advocate the so-called Choquet integral as a tool for learning monotone nonlinear models. While being widely used as a flexible aggregation operator in different fields, such as multiple criteria decision making, the Choquet integral is much less known in machine learning so far. Apart from combining monotonicity and flexibility in a mathematically sound and elegant manner, the Choquet integral has additional features making it attractive from a machine learning point of view. Notably, it offers measures for quantifying the importance of individual predictor variables and the interaction between groups of variables. Analyzing the Choquet integral from a classification perspective, we provide upper and lower bounds on its VC-dimension. Moreover, as a methodological contribution, we propose a generalization of logistic regression. The basic idea of our approach, referred to as choquistic regression, is to replace the linear function of predictor variables, which is commonly used in logistic regression to model the log odds of the positive class, by the Choquet integral. First experimental results are quite promising and suggest that the combination of monotonicity and flexibility offered by the Choquet integral facilitates strong performance in practical applications.

show abstract

“…Meanwhile, several machine learning algorithms have been modified so as to guarantee monotonicity in attributes, including nearest neighbor classification [15], decision tree learning [16] and rule induction [17]. Instead of modifying models and algorithms, one can also modify the data.…”

Section: Related Workmentioning

confidence: 99%

Choquistic Regression: Generalizing Logistic Regression using the Choquet Integral

Tehrani¹,

Cheng²,

Hüllermeier³

2011

Proceedings of the 7th Conference of the European Society for Fuzzy Logic and Technology (EUSFLAT-2011)

View full text Add to dashboard Cite

In this paper, we propose a generalization of logistic regression based on the Choquet integral. The basic idea of our approach, referred to as choquistic regression, is to replace the linear function of predictor variables, which is commonly used in logistic regression to model the log odds of the positive class, by the Choquet integral. Thus, it becomes possible to capture non-linear dependencies and interactions among predictor variables while preserving two important properties of logistic regression, namely the comprehensibility of the model and the possibility to ensure its monotonicity in individual predictors. In experimental studies with real and benchmark data, choquistic regression consistently improves upon standard logistic regression in terms of predictive accuracy.

show abstract

Nearest Neighbour Classification with Monotonicity Constraints

Cited by 76 publications

References 19 publications

Using an Ant Colony Optimization Algorithm for Monotonic Regression Rule Discovery

Using an Ant Colony Optimization Algorithm for Monotonic Regression Rule Discovery

Learning monotone nonlinear models using the Choquet integral

Choquistic Regression: Generalizing Logistic Regression using the Choquet Integral

Contact Info

Product

Resources

About