Laplace mixture of linear experts

Nguyen, Hien D.; McLachlan, Geoffrey J.

doi:10.1016/j.csda.2014.10.016

Cited by 49 publications

(53 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The data were generated exactly in the same way as in Experiment 1, except for some observations which were generated with a probability c from a class of outliers. We considered the same class of outliers as in Nguyen and McLachlan (2016), that is, the predictor x is generated uniformly over the interval (−1, 1) and the response y is set the value −2. We apply the MoE models by setting the covariate vectors as before, that is, x = r = (1, x) T .…”

Section: Methodsmentioning

confidence: 99%

“…We evaluated the performance of proposed EM algorithm by comparing it the standard normal MoE (NMoE) model (Jacobs et al, 1991;Jordan and Jacobs, 1994) and the Laplace MoE of (Nguyen and McLachlan, 2016) 1 on both simulated and real-world data sets.…”

Section: Experimental Studymentioning

confidence: 99%

“…The data consist of n = 135 yearly measurements of the global annual temperature anomalies (in degrees C) computed using data from land meteorological stations for the period of 1882 − 2012. These data have been analyzed earlier by Hansen et al (1999Hansen et al ( , 2001 and recently by Nguyen and McLachlan (2016) by using the Laplace mixture of linear experts (LMoLE).…”

Section: Temperature Anomalies Data Setmentioning

confidence: 99%

“…The problem of sensitivity of NMoE to outliers have been considered very recently by Nguyen and McLachlan (2016) where the authors proposed a Laplace mixture of linear experts (LMoLE) for a robust modeling of non-linear regression data. The model parameters are estimated by maximizing the observed-data likelihood via a minorizationmaximization (MM) algorithm.…”

Section: Introductionmentioning

confidence: 99%

“…In the same context of regression, Song et al (2014) proposed the mixture of Laplace regressions, which has been then extended by Nguyen and McLachlan (2016) to the case of mixture of experts, by introducing the Laplace mixture of linear experts (LMoLE). However, unlike our proposed TMoE model, the regression mixture models of Wei (2012), Bai et al (2012), Ingrassia et al (2012), and Song et al (2014) do not consider conditional mixing proportions, that is, mixing proportions depending on some input variables, as in the case of mixture of experts, which we investigate here.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Robust mixture of experts modeling using thetdistribution

Chamroukhi¹

2016

Neural Networks

View full text Add to dashboard Cite

Mixture of Experts (MoE) is a popular framework for modeling heterogeneity in data for regression, classification, and clustering. For regression and cluster analyses of continuous data, MoE usually use normal experts following the Gaussian distribution. However, for a set of data containing a group or groups of observations with heavy tails or atypical observations, the use of normal experts is unsuitable and can unduly affect the fit of the MoE model. We introduce a robust MoE modeling using the t distribution. The proposed t MoE (TMoE) deals with these issues regarding heavy-tailed and noisy data. We develop a dedicated expectation-maximization (EM) algorithm to estimate the parameters of the proposed model by monotonically maximizing the observed data log-likelihood. We describe how the presented model can be used in prediction and in model-based clustering of regression data. The proposed model is validated on numerical experiments carried out on simulated data, which show the effectiveness and the robustness of the proposed model in terms of modeling non-linear regression functions as well as in model-based clustering. Then, it is applied to the real-world data of tone perception for musical data analysis, and the one of temperature anomalies for the analysis of climate change data. The obtained results show the usefulness of the TMoE model for practical applications.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Experimental Studymentioning

confidence: 99%

Section: Temperature Anomalies Data Setmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Robust mixture of experts modeling using thetdistribution

Chamroukhi¹

2016

Neural Networks

View full text Add to dashboard Cite

show abstract

Parsimonious mixture‐of‐experts based on mean mixture of multivariate normal distributions

Sepahdar

Madadi

Jamalizadeh

2022

Stat

View full text Add to dashboard Cite

The mixture‐of‐experts (MoE) paradigm attempts to learn complex models by combining several “experts” via probabilistic mixture models. Each expert in the MoE model handles a small area of the data space in which a gating function controls the data‐to‐expert assignment. The MoE framework has been used extensively in designing non‐linear models in machine learning and statistics to model the heterogeneity in data for the purpose of regression, classification and clustering. The existing MoE of multi‐target regression (MoE‐MTR) models for continuous data is based on multivariate normal distributions. However, in many practical situations, for a set of data, a group or groups of observations may exhibit asymmetric and heavy‐tailed behaviour, and inference based on symmetric distributions in such situations can unduly affect the fit of the regression model. We introduce here a novel robust multivariate non‐normal MoE model by the use of mean mixture of normal distributions. The proposed model can handle the issues of MoE‐MTR models regarding possibly skewed, heavy‐tailed and noisy data. Maximum likelihood estimates of model parameters are developed based on an expectation‐maximization (EM)‐type algorithm. Parsimony is also obtained by imposing suitable constraints on the expert dispersion matrices. The usefulness of the proposed methodology is illustrated using simulated and real data sets.

show abstract

An introduction toMajorization‐Minimizationalgorithms for machine learning and statistical estimation

Nguyen

2017

WIREs Data Min & Knowl

Self Cite

View full text Add to dashboard Cite

MM (majorization-minimization) algorithms are an increasingly popular tool for solving optimization problems in machine learning and statistical estimation. This article introduces the MM algorithm framework in general and via three commonly considered example applications: Gaussian mixture regressions, multinomial logistic regressions, and support vector machines. Specific algorithms for these three examples are derived and Mathematical Programming Series A numerical demonstrations are presented. Theoretical and practical aspects of MM algorithm design are discussed.

show abstract

Laplace mixture of linear experts

Cited by 49 publications

References 32 publications

Robust mixture of experts modeling using thetdistribution

Robust mixture of experts modeling using thetdistribution

Parsimonious mixture‐of‐experts based on mean mixture of multivariate normal distributions

An introduction toMajorization‐Minimizationalgorithms for machine learning and statistical estimation

Contact Info

Product

Resources

About