An Efficient Linearly Convergent Regularized Proximal Point Algorithm for Fused Multiple Graphical Lasso Problems

Abstract

Nowadays, analysing data from different classes or over a temporal grid has attracted a great deal of interest. As a result, various multiple graphical models for learning a collection of graphical models simultaneously have been derived by introducing sparsity in graphs and similarity across multiple graphs. This paper focuses on the fused multiple graphical Lasso model which encourages not only shared pattern of sparsity, but also shared values of edges across different graphs. For solving this model, we develop an efficient regularized proximal point algorithm, where the subproblem in each iteration of the algorithm is solved by a superlinearly convergent semismooth Newton method. To implement the semismooth Newton method, we derive an explicit expression for the generalized Jacobian of the proximal mapping of the fused multiple graphical Lasso regularizer. Unlike those widely used first order methods, our approach has heavily exploited the underlying second order information through the semismooth Newton method. This can not only accelerate the convergence of the algorithm, but also improve its robustness. The efficiency and robustness of our proposed algorithm are demonstrated by comparing with some state-of-theart methods on both synthetic and real data sets. Supplementary materials for this article are available online.

A Proximal Point Dual Newton Algorithm for Solving Group Graphical Lasso Problems

“…The experimental settings are the same as that in [34,Section 4]. We adopt the stopping criteria of PPDNA, ADMM and MGL as below.…”

“…In this part, we describe the datasets which will be used later. Since these datasets have been discussed in [34], we briefly review them for the ease of reading:…”

“…However, it is not clear to us whether the convergence analysis in [33] can be directly applied to problem (1.1) as the Hessian of the first function in the objective of the problem is not uniformly bounded on its effective domain. More recently, Zhang et al [34] applied a regularized proximal point algorithm (rPPA) to solve a fused multiple graphical Lasso (FGL) model and heavily exploited the underlying second order information through the semismooth Newton method when solving the subproblems of the rPPA. Due to the polyhedral property of the FGL regularizer, the rPPA for solving the FGL problem is proven to have an arbitrary linear convergence rate in [34].…”

“…More recently, Zhang et al [34] applied a regularized proximal point algorithm (rPPA) to solve a fused multiple graphical Lasso (FGL) model and heavily exploited the underlying second order information through the semismooth Newton method when solving the subproblems of the rPPA. Due to the polyhedral property of the FGL regularizer, the rPPA for solving the FGL problem is proven to have an arbitrary linear convergence rate in [34].…”

“…In addition, Figure 1 illustrates the structure of the decision variable Θ and the vector belonging to one group Θ [ij] . Inspired by the impressive numerical performance of the rPPA for solving the FGL model [34], we will design a proximal point dual Newton algorithm (PPDNA) for solving the GGL model. Specifically, a proximal point algorithm (PPA) [20] is applied to the primal formulation of the GGL model, and a superlinearly convergent semismooth Newton method is designed to solve the dual formulations of the PPA subproblems.…”