Learning Sparse Causal Gaussian Networks With Experimental Intervention: Regularization and Coordinate Descent

Abstract

Causal networks are graphically represented by directed acyclic graphs (DAGs). Learning causal networks from data is a challenging problem due to the size of the space of DAGs, the acyclicity constraint placed on the graphical structures, and the presence of equivalence classes. In this article, we develop an L 1 -penalized likelihood approach to estimate the structure of causal Gaussian networks. A blockwise coordinate descent algorithm, which takes advantage of the acyclicity constraint, is proposed for seeking a local maximizer of the penalized likelihood. We establish that model selection consistency for causal Gaussian networks can be achieved with the adaptive lasso penalty and sufficient experimental interventions. Simulation and real data examples are used to demonstrate the effectiveness of our method. In particular, our method shows satisfactory performance for DAGs with 200 nodes, which have about 20,000 free parameters. Supplementary materials for this article are available online.

Estimation of Directed Acyclic Graphs Through Two-Stage Adaptive Lasso for Gene Network Inference

“…Zou (2006) showed that the adaptive lasso satisfies the consistency in model selection if à ij is a $\sqrt{n}$-consistent estimate of A ij and suggested using the ordinary least squares (OLS) estimate for à ij . Fu and Zhou (2013) proposed the ordinary least squares (OLS) estimate with an upper bound, which is $false(\frac{1}{false|{Ã}_{ij}{|}^{\gamma }},\frac{1}{{false(N^{-1}false)}^{\gamma }}false)$ with N = 10 4 . However, if there are correlations among variables, the estimates from OLS are unstable.…”

“…The formula (8) provides the lower and upper bound of 1 and N γ , respectively. In our simulation study, we use N = 10 4 as Fu and Zhou (2013) did. We construct the initial estimates à ij from the regular lasso estimates by minimizing function (8) with a certain λ 0 , γ, and w ij = 1.…”

“…We compared the NS-DIST method with the following five recent DAG estimation methods: the PC-stable method provided by Colombo and Maathuis (2013), the MMHC method provided by Tsamardinos et al (2006), the GES method provided by Chickering (2002b), the CD algorithm provided by Fu and Zhou (2013), and a permutation approach with the Lasso framework provided by Shojaie and Michailidis (2010). As proposed by Fu and Zhou (2013), we use γ = 0.15 for the CD method.…”

“…In addition, Zou (2006) proposed the adaptive lasso regression, which gives asymptotic consistency in variable selection. Fu and Zhou (2013) proposed a profiled likelihood with an adaptive lasso penalty to estimate DAGs under unknown variable order based on experimental data with interventions, and used a blockwise coordinate descent (CD) algorithm to find a local optimal solution. Aragam and Zhou (2015) improved the CD algorithm under the observational data.…”

Identification of Biomarkers for Predicting the Overall Survival of Ovarian Cancer Patients: a Sparse Group Lasso Approach

“…However, it tends to yields a large number of false positives in the sparse network problem, as pointed out by Fu and Zhou in their seminal paper (Fu & Zhou, 2013). Fu and Zhou proposed an "elbow method" that outperforms the cross-validation method, where the optimal tuning parameter corresponds to the change point at which an increase of λ does not yield a substantial decrease of log-likelihood.…”