Are fit indices used to test psychopathology structure biased? A simulation study.

Greene, Ashley L.; Eaton, Nicholas R.; Li, Kaiqiao; Forbes, Miriam K.; Krueger, Robert F.; Markon, Kristian E.; Waldman, Irwin D.; Cicero, David C.; Conway, Christopher; Docherty, Anna R.; Fried, Eiko I.; Ivanova, Masha Y.; Jonas, Katherine; Latzman, Robert D.; Patrick, Christopher J.; Reininghaus, Ulrich; Tackett, Jennifer L.; Wright, Aidan G.C.; Kotov, Roman

doi:10.1037/abn0000434

Cited by 132 publications

(215 citation statements)

References 137 publications

(294 reference statements)

Supporting

Mentioning

205

Contrasting

Order By: Relevance

“…First, the problem of statistical equivalence also plays out within the model families listed above, not only across the families. For example, (Greene et al, 2019), establishing the same inferential problems as discussed above. Second, temporal data can help eliminate some equivalent models, since it makes little sense to orient arrows backwards in time, but they do not eliminate the problem of statistical equivalence in principle (Raykov & Marcoulides, 2001).…”

Section: Statistical Equivalence and The Inference Gapmentioning

confidence: 92%

Lack of theory building and testing impedes progress in the factor and network literature

Fried¹

2020

Preprint

113

138

View full text Add to dashboard Cite

The last decade has brought reforms to improve methodological practices, with the goal to increase the reliability and replicability of effects. However, explanations of effects remain scarce, and a growing chorus of scholars argues that the replicability crisis has distracted from a crisis of theory. In the same decade, the empirical literature using factor and network models has grown rapidly. I discuss three ways in which this literature falls short of theory building and testing. First, statistical and theoretical models are conflated, leading to invalid inferences such as the existence of psychological constructs based on factor models, or recommendations for clinical interventions based on network models. I demonstrate this inferential gap in a simulation study on statistical equivalence: excellent model fit does little to corroborate a theory, regardless of quality or quantity of data. Second, researchers fail to explicate theories about psychological constructs, but use implicit causal beliefs to guide inferences. These latent theories have led to problematic best practices in psychological research where inferences are drawn based on one specific causal model that is assumed, but not explicated. Third, explicated theories are often weak theories: narrative and imprecise descriptions vulnerable to hidden assumptions and unknowns. They fail to make clear predictions, and it remains unclear whether statistical effects corroborate such theories or not. Weak theories are immune to refutation or revision. I argue that these three challenges to theory building and testing are common and harmful, and impede theory formation, failure, and reform. A renewed focus on theoretical psychology and formal models offers a way forward.

show abstract

Section: Statistical Equivalence and The Inference Gapmentioning

confidence: 92%

Lack of theory building and testing impedes progress in the factor and network literature

Fried¹

2020

Preprint

113

138

View full text Add to dashboard Cite

show abstract

“…The bifactor model's flexibility can also enable it to show superior global fit than alternatives, even when the other models were themselves used to simulate data (33,(38)(39)(40)(41)(42)(43)(44)(45)(46). For example, skewed item distributions and unmodeled cross-loadings or correlated residuals can all lead fit statistics to favor the bifactor model over a correlated-factors model (with no general factor), even if the correlatedfactors model more accurately describes the true structure (46,47). The bifactor model's flexibility can also result in good model-data fit even when used with very noisy data or nonsense response patterns (33,45).…”

Section: Biological Psychiatrymentioning

confidence: 99%

Appropriate Use of Bifactor Analysis in Psychopathology Research: Appreciating Benefits and Limitations

Bornovalova

Choate

Fatimah

et al. 2020

Biological Psychiatry

154

129

View full text Add to dashboard Cite

Co-occurrence of psychiatric disorders is welldocumented. Recent quantitative efforts have moved toward an understanding of this phenomenon, with the 'general psychopathology' or p-factor model emerging as the most prominent characterization. Over the past decade, bifactor model analysis has become increasingly popular as a statistical approach to describe common/shared and unique elements in psychopathology. However, recent work has highlighted potential problems with common approaches to evaluating and interpreting bifactor models. Here, we argue that, when properly applied and interpreted, bifactor models can be useful for answering some important questions in psychology and psychiatry research. We review problems with evaluating bifactor models based on global model fit statistics. We then describe more valid approaches to evaluating bifactor models and highlight three types of research questions for which bifactor models are wellsuited to answer. We also discuss the utility and limits of bifactor applications in genetic and neurobiological research. We close by comparing advantages and disadvantages of bifactor models to other analytic approaches and noting that no statistical model is a panacea to rectify limitations of the research design used to gather data. 1 Sometimes, group factors are called "specific factors." However, "specific factor" more correctly refers to an item's reliable (non-error) variance that is not shared with other items (5).

show abstract

“…Each model fit the data well (Table S10). Because of difficulties in choosing between substantively different models that fit the data well using fit statistics, we evaluated the two structural models in terms of their criterion validity (Bonifay et al, 2017;Greene et al, 2019). Table 1, all factors in both models had acceptable H indices >0.70 (Hancock & Mueller, 2001) and each specific factor in the bifactor models was reliable according to omega statistics.…”

Section: Confirmatory Factor Analyses In the Second Half Of The Samplementioning

confidence: 99%

Criterion Validity and Relationships between Alternative Hierarchical Dimensional Models of General and Specific Psychopathology

Moore

Kaczkurkin

Durham

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Psychopathology can be viewed as a hierarchy of correlated dimensions. Many studies have supported this conceptualization, but they have used alternative statistical models with differing interpretations. In bifactor models, every symptom loads on both the general factor and one specific factor (e.g., internalizing), which partitions the total explained variance in each symptom between these orthogonal factors. In second-order models, symptoms load on one of several correlated lower-order factors. These lower-order factors load on a second-order general factor, which is defined by the variance shared by the lower-order factors. Thus, the factors in secondorder models are not orthogonal. Choosing between these valid statistical models depends on the hypothesis being tested. Because bifactor models define orthogonal phenotypes with distinct sources of variance, they are optimal for studies of shared and unique associations of the dimensions of psychopathology with external variables putatively relevant to etiology and mechanisms. Concerns have been raised, however, about the reliability of the orthogonal specific factors in bifactor models. We evaluated this concern using parent symptom ratings of 9-10 year olds in the ABCD Study. Psychometric indices indicated that all factors in both bifactor and second-order models exhibited at least adequate construct reliability and estimated replicability.The factors defined in bifactor and second-order models were highly to moderately correlated across models, but have different interpretations. All factors in both models demonstrated significant associations with external criterion variables of theoretical and clinical importance, but the interpretation of such associations in second-order models was ambiguous due to shared variance among factors.

show abstract

Are fit indices used to test psychopathology structure biased? A simulation study.

Cited by 132 publications

References 137 publications

Lack of theory building and testing impedes progress in the factor and network literature

Lack of theory building and testing impedes progress in the factor and network literature

Appropriate Use of Bifactor Analysis in Psychopathology Research: Appreciating Benefits and Limitations

Criterion Validity and Relationships between Alternative Hierarchical Dimensional Models of General and Specific Psychopathology

Contact Info

Product

Resources

About