On the Importance of Estimating Parameter Uncertainty in Network Psychometrics: A Response to Forbes et al. (2019)

Abstract

In their recent paper, Forbes et al. (2019; FWMK) evaluate the replicability of network models in two studies. They identify considerable replicability issues, concluding that "current 'stateof-the-art' methods in the psychopathology network literature [ … ] are not well-suited to analyzing the structure of the relationships between individual symptoms". Such strong claims require strong evidence, which the authors do not provide. FWMK identify low replicability by analyzing point estimates of networks; contrast low replicability with results of two statistical tests that indicate higher replicability, and conclude that these tests are problematic. We make four points. First, statistical tests are superior to the visual inspection of point estimates, because tests take into account sampling variability. Second, FWMK misinterpret the statistical tests in several important ways. Third, FWMK did not follow established recommendations when estimating networks in their first study, underestimating replicability. Fourth, FWMK draw conclusions about methodology, which does not follow from investigations of data, and requires investigations of methodology. Overall, we show that the "poor replicability "observed by FWMK occurs due to sampling variability and use of suboptimal methods. We conclude by discussing important recent simulation work that guides researchers to use models appropriate for their data, such as nonregularized estimation routines.

Meta-analytic Gaussian Network Aggregation

“…Even though Fried et al (2018) based their main analyses on polychoric correlation matrices because the data were assessed on a likert scale, we make use of the Pearson correlation matrices provided in their supplementary materials. We do this because polychoric correlations do not evaluate to the likelihood of the data, and because polychoric correlations have been found to be quite unstable in network estimation at lower sample sizes (Fried et al 2021) We estimated a GGM using significance pruning for each dataset individually in the same way as described in Sect. 3.2, Supplement 1 and Supplement 2.1.…”

The network of stress-related states and depression and anxiety symptoms during the COVID-19 lockdown

“…Consequently, Spearman correlations were used to estimate the network. Given the large number of participants in the present study, an unregularized method referred to as the Graphical Gaussian Model ModSelect Algorithm (i.e., ggmModSelect) in the R-package qgraph ( Epskamp et al., 2012 ) was used in line with recent recommendations ( Fried et al., 2020 ; Williams et al., 2019 ). In this procedure (see online supplementary material for R code), the graphical least absolute shrinkage and selection operator (gLASSO) is used to estimate the structure of 100 regularized network models ranging from sparse to dense.…”

Comorbidity Between Depression and Anxiety in Adolescents: Bridge Symptoms and Relevance of Risk and Protective Factors

“…Firstly, network analysis has been criticized on a number of grounds, including failure to replicate within and between samples, being non-causal, and that edges can result by chance (Forbes et al 2017a;Steinley et al 2017;Forbes et al 2021). However, these critics themselves, have been criticized, on a number of grounds, including estimating unsuitable network models for the given data, using different network models that result in different network structures, and ignoring the impact of sampling variability on the results of network models (Borsboom et al 2017;Epskamp et al 2018a, b;Fried et al 2020;Jones et al 2020). Secondly, the data analyzed here is cross-sectional as these risk and protective factors, and symptoms of anxiety and depression were present together only at age 13.…”