A majorization-minimization algorithm for (multiple) hyperparameter learning

Foo, Chuan-Sheng; Chuong, B.; Ng, Andrew Y.

doi:10.1145/1553374.1553415

Cited by 23 publications

(23 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We use the Bayesian regularization method in [11] for the setting of the regularization parameters (i.e., λ 1 and λ 2 ) in all models. The core idea of [11] is to first place a Gamma prior on each of the regularization parameters and then integrate out the regularization parameters.…”

Section: Experimental Settingsmentioning

confidence: 99%

“…The core idea of [11] is to first place a Gamma prior on each of the regularization parameters and then integrate out the regularization parameters. By using the majorization-minimization (MM) algorithm [14], [15], [22], the objective function in each iteration is similar to the original problem with the regularization parameters inversely depending on the solution in the previous iteration.…”

Section: Experimental Settingsmentioning

confidence: 99%

See 1 more Smart Citation

A Unified Framework for Epidemic Prediction based on Poisson Regression

Zhang

Cheung

Liu

2015

IEEE Trans. Knowl. Data Eng.

View full text Add to dashboard Cite

Epidemic prediction is an important problem in epidemic control. Poisson regression methods are often adopted in existing works, mostly with only the (intra-)regional environmental factors considered. As the diffusion of epidemics is affected by not only the intra-regional factors but also inter-regional and external ones, a unified framework based on Poisson regression with the three types of factors incorporated is proposed for the prediction. Specifically, we propose a Poisson-regression-based model first with the intra-regional and inter-regional factors included. The intra-regional factor in a particular time interval is represented by one feature vector with the regionally environmental and social factors considered. The inter-regional factor is modeled by a diffusion matrix which describes the possibilities that the epidemics can spread from one region to another, which in turn accounts for the propagating effects of the infected cases. To learn the structure of the diffusion matrix, we propose two approaches -utilizing some a priori knowledge (e.g., transportation network) and estimating it from scratch via a sparse structure assumption. The resulting optimization problem of the maximum a posterior solution is a convex one and can be efficiently solved by the alternating direction method of multipliers (ADMM). In addition, we incorporate also the external factor, i.e., the imported cases. With one fact that the distribution of the number of infected cases over a year is (approximately) unimodal for most epidemics and one assumption that the importing rate has a small variance over the year, we can approximate the effect of the external factor with a parametric function (e.g., a quadratic function) over time. The resulting optimization problem is still convex and can be also solved by the ADMM algorithm. Empirical evaluations are conducted based on a real data set which records the 16-days-reported cases in the Yunnan province of China for seven years, from 2005 to 2011. The experimental results demonstrate the effectiveness of our proposed models.

show abstract

Section: Experimental Settingsmentioning

confidence: 99%

Section: Experimental Settingsmentioning

confidence: 99%

A Unified Framework for Epidemic Prediction based on Poisson Regression

Zhang

Cheung

Liu

2015

IEEE Trans. Knowl. Data Eng.

View full text Add to dashboard Cite

show abstract

“…In the field of machine learning, MM algorithm has been applied to parameters selection for bayesian classification [23]. In the area of signal processing, MM algorithm leads to a number of interesting applications, including wavelet-based processing [24] and total variation (TV) minimization [25].…”

Section: B MM Algorithm and Reweighted Approachesmentioning

confidence: 99%

Low-Rank Structure Learning via Nonconvex Heuristic Recovery

Deng

Dai

Liu

et al. 2013

IEEE Trans. Neural Netw. Learning Syst.

106

View full text Add to dashboard Cite

Abstract-Recovering intrinsic data structure from corrupted observations plays an important role in various tasks in the communities of machine learning and signal processing. In this paper, we propose a novel model, named log-sum heuristic recovery (LHR), to learn the essential low-rank structure from corrupted data. Different from traditional approaches, which directly utilize ℓ1 norm to measure the sparseness, LHR introduces a more reasonable log-sum measurement to enhance the sparsity in both the intrinsic low-rank structure and in the sparse corruptions. Although the proposed LHR optimization is no longer convex, it still can be effectively solved by a majorizationminimization (MM) type algorithm, with which the non-convex objective function is iteratively replaced by its convex surrogate and LHR finally falls into the general framework of reweighed approaches. We prove that the MM-type algorithm can converge to a stationary point after successive iteration. We test the performance of our proposed model by applying it to solve two typical problems: robust principal component analysis (RPCA) and low-rank representation (LRR). For RPCA, we compare LHR with the benchmark Principal Component Pursuit (PCP) method from both the perspectives of simulations and practical applications. For LRR, we apply LHR to compute the lowrank representation matrix for motion segmentation and stock clustering. Experimental results on low rank structure learning demonstrate that the proposed Log-sum based model performs much better than the ℓ1-based method on for data with higher rank and with denser corruptions.Index Terms-Log-sum heuristic, compressive sensing, sparse optimization, matrix learning, nuclear norm minimization.

show abstract

“…We note that, to preserve the posterior covariance matrix, these parameters are not integrated out as discussed in [10].…”

Section: E Learning Additional Model Parametersmentioning

confidence: 99%

Active Learning and Basis Selection for Kernel-Based Linear Models: A Bayesian Perspective

Paisley

Liao

Carin

2010

IEEE Trans. Signal Process.

View full text Add to dashboard Cite

We develop an active learning algorithm for kernel-based linear regression and classification. The proposed greedy algorithm employs a minimum-entropy criterion derived using a Bayesian interpretation of ridge regression. We assume access to a matrix, Φ ∈ R N ×N , for which the (i, j) th element is defined by the kernel function K(γ i , γ j ) ∈ R, with the observed datawhere y i is a real-valued response or integer-valued label, which we do not have access to a priori. To achieve this goal, a sub-matrix, Φ I l ,I b ∈ R n×m , is sought that corresponds to the intersection of n rows and m columns of Φ, indexed by the sets I l and I b respectively. Typically m N and n N . and (ii) using active learning, sequentially determine which subset of n elements of {y i } N i=1 should be acquired; both stopping values, |I b | = m and |I l | = n, are also to be inferred from the data. These steps are taken with the goal of minimizing the uncertainty of the model parameters, x, as measured by the differential entropy of its posterior distribution. The parameter vector x ∈ R m , as well as the model bias η ∈ R, is then learned from the resulting problem, y I l = Φ I l ,I b x + η1 + . The remaining N − n responses/labels not included in y I l can be inferred by applying x to the remaining N − n rows of Φ :,I b .We show experimental results for several regression and classification problems, and compare with other active learning methods.

show abstract

A majorization-minimization algorithm for (multiple) hyperparameter learning

Cited by 23 publications

References 18 publications

A Unified Framework for Epidemic Prediction based on Poisson Regression

A Unified Framework for Epidemic Prediction based on Poisson Regression

Low-Rank Structure Learning via Nonconvex Heuristic Recovery

Active Learning and Basis Selection for Kernel-Based Linear Models: A Bayesian Perspective

Contact Info

Product

Resources

About