Finite mixture models for regression problems

Abstract

Finite mixture models (FMMs) are a ubiquitous tool for the analysis of heterogeneous data across a broad number of fields including agriculture, bioinformatics, botany, cell biology, economics, fisheries research, genetics, genomics, geology, machine learning, medicine, palaeontology, psychology, and zoology, among many others. Due to their flexibility, FMMs can be used to cluster data, classify data, estimate densities, and increasingly, they are also being used to conduct regression analysis and to analyze regression outcomes. There is now an expansive literature on the usage of FMMs for regression, as well as a broad demand for the development of such methods for the analysis of new and complex data.This thesis begins with a summary of the current literature on FMMs and their applications to regression problems. Here, the mixture of regression models (MRMs), cluster-weight models (CWMs), mixtures of experts (MoEs), and mixtures of linear of mixed effects models (MLMMs), as well as other variants of FMMs for regression analysis are introduced. Various properties such as denseness and identifiability, as well as maximum likelihood (ML) estimation techniques such as the expectation-maximization (EM) and minorization-maximization (MM) algorithms are discussed, and a review is presented regarding asymptotic inference and model selection in FMMs. A new result on the characterization of a t linear CWM (LCWM) is also presented. Some new applications of FMMs to regression problems are then discussed.Firstly, a series of models based on FMMs are presented for the clustering and classification of sparsely sampled bivariate functional data. These methods are named mixture of spatial spline regression (MSSR) and MSSR discriminant analysis (MSSRDA). MSSR is constructed using the theory of MLMMs and spatial splines, and an EM algorithm for the ML estimation of the model is presented. MSSRDA is then constructed by combining MSSR with the mixture discriminant analysis framework for classification. The methods are tested on their ability to cluster and classify simulated data. An example application to handwritten digits recognition is then presented. Here, it is shown that MSSR and MSSRDA perform comparably to currently available methods, and outperform said methods in missing data scenarios.Secondly, an FMM is used to produce a false discovery rate (FDR) control procedure for magnetic resonance imaging (MRI) data. In MRI data analysis, millions of hypotheses are often tested simultaneously, resulting in inflated numbers of false positive results. Many of the available FDR techniques for MRI data either do not take into account the spatial structure or rely on difficult to verify assumptions and user-specified parameters. To address these shortcomings, the Markov random field (MRF) FDR (MRF-FDR) technique is presented. MRF-FDR uses a Gaussian mixture model (GMM) to perform FDR control based on empirical-Bayesian principles. An MRF is then used to make the outcome of the GMM spatially coherent. The MRF is fitted using ...

Methods and Applications of Clusterwise Linear Regression: A Survey and Comparison

“…These models are formulated using the conditional mixture and maximum likelihood methodology. The CLR models based on the maximum likelihood methodology are also known as finite mixture models for regression problems [74] and finite mixtures of linear regression [39]. Finite mixture models for regression were discussed in [81].…”

Mixtures of Semi-Parametric Generalised Linear Models

“…In this paper we start by considering MGLM [13,14], proposing a semi-parametric estimation of the component inverse link functions for modelling the component conditional expected value of the response variable given explanatory variables. The proposed MSPGLM has several important advantages: In [5], it is shown that under fairly general conditions, √ n consistent estimates of the component parameter directions are obtained.…”