Mr. Kim et al. Reply

Building theories on top of, and not independent of, statistical models: The case of the p-factor

“…Notwithstanding concerns over interpretability of these and other specific factors (Bagby et al, 2007;Vanheule et al, 2008), a natural consequence of poorly defined specific factors is that they are inconsistent across studies and sometimes unreliable. We have raised concerns about poor factor reliability in bifactor models of psychopathology in other work Watts et al, 2019;see also Forbes et al, under review;Kim, Greene, et al, 2019), and colleagues (2016a, 2016b) noted this tendency of the bifactor model more generally. Of the 15 studies included in our review, 80 percent reported one or more specific factors with standardized loadings of less than .20, and many of these loadings were near 0 or even negative.…”

“…Bias is present when model fit indicates a preference for a model other than the true P-FACTOR THEORY 7 structure. Bifactor model fit bias has been well-established in the intelligence literature (e.g., Murray & Johnson, 2013) and was more recently extended to the psychopathology literature (Greene et al, 2019). Greene and colleagues (2019) showed that model fit more often than not could not identify the true structure.…”

Three recommendations based on a comparison of the reliability and validity of the predominant models used in research on the empirical structure of psychopathology.

“…Psychopathology research has begun making important inroads into this issue—for example, comparing a subset of models (e.g., bifactor, correlated factors model) on the basis of model-based reliability indices recommended by Rodriguez et al (2016b; see Conway et al, 2019; Gomez et al, 2019; Watts et al, 2019). These recent developments in the literature have highlighted diverging levels of interpretability and factor reliability among structural models of psychopathology (e.g., occasionally weak or negative specific factor loadings in a bifactor model that are at odds with the interpretation of the corresponding factor in a correlated factor model, and greater reliability for correlated factors compared to specific factors from a bifactor model; Eid et al, 2017; Kim et al, 2019; Watts et al, 2019). Our study will use complementary metrics in addition to traditional fit indices to determine the degree to which the “best fitting” model also corresponds to superior factor strength and reliability.…”