Stable graphical model estimation with Random Forests for discrete, continuous, and mixed variables

Fellinghauer, Bernd A. G.; Bühlmann, Peter; Ryffel, Martin; Rhein, Michael von; Reinhardt, Jan D.

doi:10.1016/j.csda.2013.02.022

Cited by 71 publications

(120 citation statements)

References 42 publications

Supporting

Mentioning

112

Contrasting

Unclassified

Order By: Relevance

“…Stability selection has since been widely used, e.g. for gene regulatory network analysis [19,20], in genome-wide association studies [21], graphical models [22,23] or even in ecology [24]. In most publications, stability selection is used in combination with lasso or similar penalization approaches.…”

Section: Introductionmentioning

confidence: 99%

Controlling false discoveries in high-dimensional situations: boosting with stability selection

2015

View full text Add to dashboard Cite

BackgroundModern biotechnologies often result in high-dimensional data sets with many more variables than observations (n≪p). These data sets pose new challenges to statistical analysis: Variable selection becomes one of the most important tasks in this setting. Similar challenges arise if in modern data sets from observational studies, e.g., in ecology, where flexible, non-linear models are fitted to high-dimensional data. We assess the recently proposed flexible framework for variable selection called stability selection. By the use of resampling procedures, stability selection adds a finite sample error control to high-dimensional variable selection procedures such as Lasso or boosting. We consider the combination of boosting and stability selection and present results from a detailed simulation study that provide insights into the usefulness of this combination. The interpretation of the used error bounds is elaborated and insights for practical data analysis are given.ResultsStability selection with boosting was able to detect influential predictors in high-dimensional settings while controlling the given error bound in various simulation scenarios. The dependence on various parameters such as the sample size, the number of truly influential variables or tuning parameters of the algorithm was investigated. The results were applied to investigate phenotype measurements in patients with autism spectrum disorders using a log-linear interaction model which was fitted by boosting. Stability selection identified five differentially expressed amino acid pathways.ConclusionStability selection is implemented in the freely available R package stabs (http://CRAN.R-project.org/package=stabs). It proved to work well in high-dimensional settings with more predictors than observations for both, linear and additive models. The original version of stability selection, which controls the per-family error rate, is quite conservative, though, this is much less the case for its improvement, complementary pairs stability selection. Nevertheless, care should be taken to appropriately specify the error bound.Electronic supplementary materialThe online version of this article (doi:10.1186/s12859-015-0575-3) contains supplementary material, which is available to authorized users.

show abstract

Section: Introductionmentioning

confidence: 99%

Controlling false discoveries in high-dimensional situations: boosting with stability selection

2015

View full text Add to dashboard Cite

show abstract

“…In [2], the authors proposed a simplification of the CG model using a group lasso type sparsity penalty to work with high-dimensional data. Both [13] and [7] study special cases of the CG model that significantly reduce the number of free parameters, enabling scalability to larger datasets. We chose to focus upon the model from [13] due to its efficiency and its success in prior applications to biological data [23].…”

Section: Introductionmentioning

confidence: 99%

Comparison of strategies for scalable causal discovery of latent variable models from mixed data

Raghu

Ramsey

Morris

et al. 2018

Int J Data Sci Anal

View full text Add to dashboard Cite

Modern technologies allow large, complex biomedical datasets to be collected from patient cohorts. These datasets are comprised of both continuous and categorical data (“Mixed Data”), and essential variables may be unobserved in this data due to the complex nature of biomedical phenomena. Causal inference algorithms can identify important relationships from biomedical data; however, handling the challenges of causal inference over mixed data with unmeasured confounders in a scalable way is still an open problem. Despite recent advances into causal discovery strategies that could potentially handle these challenges; individually, no study currently exists that comprehensively compares these approaches in this setting. In this paper, we present a comparative study that addresses this problem by comparing the accuracy and efficiency of different strategies in large, mixed datasets with latent confounders. We experiment with two extensions of the Fast Causal Inference algorithm: a maximum probability search procedure we recently developed to identify causal orientations more accurately, and a strategy which quickly eliminates unlikely adjacencies in order to achieve scalability to high-dimensional data. We demonstrate that these methods significantly outperform the state of the art in the field by achieving both accurate edge orientations and tractable running time in simulation experiments on datasets with up to 500 variables. Finally, we demonstrate the usability of the best performing approach on real data by applying it to a biomedical dataset of HIV-infected individuals.Electronic supplementary materialThe online version of this article (10.1007/s41060-018-0104-3) contains supplementary material, which is available to authorized users.

show abstract

“…Recently, Edwards et al [13] proposed to estimate stable graphical models with random forest in combination with stability selection using regression models. Their main idea is motivated by the following theorem.…”

Section: Regression-type Graphical Modelsmentioning

confidence: 99%

“…In this paper, we present two classes of graphical models, namely strongly decomposable graphical models [12] and regression-type graphical models [13], which are classes of models that can be used for analyzing high-dimensional data with mixed variables. Assuming that the conditional distribution of a variable A given the rest depends on any realization of the remaining variables only through the conditional mean function, the regression models are useful to find the matrix weights which can be further employed to recover the network.…”

Section: Introductionmentioning

confidence: 99%

Inferring Networks from High-Dimensional Data with Mixed Variables

Abbruzzo¹,

Mineo²

2014

Contributions to Statistics

View full text Add to dashboard Cite

We present two methodologies to deal with high-dimensional data with mixed variables, the strongly decomposable graphical model and the regression-type graphical model. The first model is used to infer conditional independence graphs. The latter model is applied to compute the relative importance or contribution of each predictor to the response variables. Recently, penalized likelihood approaches have also been proposed to estimate graph structures. In a simulation study, we compare the performance of the strongly decomposable graphical model and the graphical lasso in terms of graph recovering. Five different graph structures are used to simulate the data: the banded graph, the cluster graph, the random graph, the hub graph and the scale-free graph. We assume the graphs are sparse. Our finding, in the simulation study, is that the strongly decomposable graphical model shows, generally, comparable or better performance both in low and high-dimensional case. Finally, we show an application on mixed data.

show abstract

Stable graphical model estimation with Random Forests for discrete, continuous, and mixed variables

Cited by 71 publications

References 42 publications

Controlling false discoveries in high-dimensional situations: boosting with stability selection

Controlling false discoveries in high-dimensional situations: boosting with stability selection

Comparison of strategies for scalable causal discovery of latent variable models from mixed data

Inferring Networks from High-Dimensional Data with Mixed Variables

Contact Info

Product

Resources

About