Network exploration via the adaptive LASSO and SCAD penalties

Abstract

Graphical models are frequently used to explore networks, such as genetic networks, among a set of variables. This is usually carried out via exploring the sparsity of the precision matrix of the variables under consideration. Penalized likelihood methods are often used in such explorations. Yet, positive-definiteness constraints of precision matrices make the optimization problem challenging. We introduce non-concave penalties and the adaptive LASSO penalty to attenuate the bias problem in the network estimation. Through the local linear approximation to the non-concave penalty functions, the problem of precision matrix estimation is recast as a sequence of penalized likelihood problems with a weighted L1 penalty and solved using the efficient algorithm of Friedman et al. (2008). Our estimation schemes are applied to two real datasets. Simulation experiments and asymptotic theory are used to justify our proposed methods.

Learning Sparse Causal Gaussian Networks With Experimental Intervention: Regularization and Coordinate Descent

“…We follow arguments similar to those given by Fan, Feng, and Wu (2009) to prove Theorems 2 and 3. However, one cannot directly apply Fan et al's results here, because the parameters must satisfy the acyclicity constraint, the data we have are not iid observations due to interventions, and the identifiability of a DAG is not always guaranteed.…”

Learning a common substructure of multiple graphical Gaussian models

“…Several future research topics have been indicated, including analyzing the asymptotic property of the CSSL problem (8), and extending the current formulation to the adaptive Lasso (Zou, 2006;Fan et al, 2009) type one to guarantee the oracle property (Zou, 2006) of the estimator. Applying the notion of commonness to more general dependency models, such as those with non-linear relations or commonness based on higher-order moment statistics, is also important.…”

An inexact interior point method for L 1-regularized sparse covariance selection

“…Once they are obtained, ∆u, ∆Z can be obtained from (12) and the first equation of (10), respectively. After that, ∆x 1 , ∆x 2 can be computed from (11).…”

“…Once ∆y, ∆u are computed, ∆Z can be obtained from the first equation of (10), while ∆x 1 , ∆x 2 can be computed from (11). The unknown ∆X is easy to obtain since from the fourth equation of (10), we have…”

“…and ∆x 1 , ∆x 2 can be computed from (11). The unknown ∆X can be computed either from (22) or from the following equation: ∆X = A T r p + B T (g − q • ∆u).…”