Comparing Estimation Methods for Psychometric Networks with Ordinal Data

Abstract

Ordinal data are extremely common in psychological research, with variables often assessed using Likert-type scales that take on only a few values. At the same time, researchers are increasingly fitting network models to ordinal item-level data. Yet very little work has evaluated how network estimation techniques perform when data are ordinal. We use a Monte Carlo simulation to evaluate and compare the performance of three estimation methods applied to either Pearson or polychoric correlations: EBIC graphical lasso with regularized edge estimates (“EBIC”), BIC model selection with partial correlation edge estimates (“BIC”), and multiple regression with p-value-based edge selection and partial correlation edge estimates (“MR”). We vary the number and distribution of thresholds, distribution of the underlying continuous data, sample size, model size, and network density, and we evaluate results in terms of model structure (sensitivity and false positive rate) and edge weight bias. Our results show that the effect of treating the data as ordinal versus continuous depends primarily on the number of levels in the data, and that estimation performance was affected by the sample size, the shape of the underlying distribution, and the symmetry of underlying thresholds. Furthermore, which estimation method is recommended depends on the research goals: MR methods tended to maximize sensitivity of edge detection, BIC approaches minimized false positives, and either one of these produced accurate edge weight estimates in sufficiently large samples. We identify some particularly difficult combinations of conditions for which no method produces stable results.

Network analysis of multivariate data in psychological science

“…First, although treatments of dichotomous, unordered categorical and continuous data and their combinations are well developed 57 , treatments of ordinal data are still suboptimal. Ongoing research is developing approaches for such data, which are common in the social sciences 92,93 . Second, estimation routines have traditionally used nodewise regularized regression 16 or the graphical lasso 33 .…”

Network analysis of multivariate data in psychological science

“…First, although treatments of dichotomous, unordered categorical and continuous data and their combinations are well developed 57 , treatments of ordinal data are still suboptimal. Ongoing research is developing approaches for such data, which are common in the social sciences 92,93…”

Partial Identification of Latent Correlations with Ordinal Data

“…When the data is a random sample of vector variables with ordinal coordinates, it is usually inappropriate to estimate structural equation models directly on the covariance matrix of the observations (Bollen, 1989, Chapter 9). Instead, the correlation matrix of a latent continuous random vector Z is used as input for the models, such as ordinal factor analysis (Christoffersson, 1975;Muthén, 1978), ordinal principal component analysis (Kolenikov & Angeles, 2009), ordinal structural equation models (Jöreskog, 1984;Muthén, 1994), and, more recently, ordinal methods in network psychometrics (Epskamp, 2017;Isvoranu & Epskamp, 2021;Johal & Rhemtulla, 2021).…”