Sparse Bayesian Hierarchical Mixture of Experts and Variational Inference

Iikubo, Yuji; Horii, Shunsuke; Matsushima, Toshiyasu

doi:10.23919/isita.2018.8664333

Cited by 2 publications

(3 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To attain high prediction performance with interpretability, previous works have proposed some extensions of linear regression model (e.g., [1][2][3][4][5][6][7][8][9][10][11][12][13][14]). In stratified regression [1], the data are stratified based on design variables, and at each level, a linear regression model is defined using explanatory variables and response variables.…”

Section: Introductionmentioning

confidence: 99%

“…In stratified regression [1], the data are stratified based on design variables, and at each level, a linear regression model is defined using explanatory variables and response variables. Another extension of the linear regression model is Mixture of Experts Model (e.g., [2,3]) or Hierarchical Mixture of Experts Model (e.g., [4][5][6][7][8]), which consists of some experts and a gating function. Each expert is a linear regression model that outputs the response variable.…”

Section: Introductionmentioning

confidence: 99%

“…Regarding (1), recent works (e.g., [5][6][7]) often fix the number of regression models, so the number of clusters. On the other hand, our algorithm do not choose the number of clusters to be a single value, but consider several numbers of clusters and weight their prediction results.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Cluster’s Number Free Bayes Prediction of General Framework on Mixture of Regression Models

et al. 2021

Self Cite

View full text Add to dashboard Cite

Prediction based on a single linear regression model is one of the most common way in various field of studies. It enables us to understand the structure of data, but might not be suitable to express the data whose structure is complex. To express the structure of data more accurately, we make assumption that the data can be divided in clusters, and has a linear regression model in each cluster. In this case, we can assume that each explanatory variable has their own role; explaining the assignment to the clusters, explaining the regression to the target variable, or being both of them. Introducing probabilistic structure to the data generating process, we derive the optimal prediction under Bayes criterion and the algorithm which calculates it sub-optimally with variational inference method. One of the advantages of our algorithm is that it automatically weights the probabilities of being each number of clusters in the process of the algorithm, therefore it solves the concern about selection of the number of clusters. Some experiments are performed on both synthetic and real data to demonstrate the above advantages and to discover some behaviors and tendencies of the algorithm.

show abstract