Does crude measurement contribute to  observed unidimensionality of psychological constructs? An example with DSM-5 alcohol use disorder

Watts, Ashley L.; Boness, Cassandra L.; Stevens, Jordan; Steinley, Douglas; Sher, Kenneth J.

doi:10.31234/osf.io/paxd4

Cited by 14 publications

(21 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Researchers may mistakenly interpret heterogeneous sets of items as being homogeneous in nature if model fit is "good" or "acceptable" for single-factor models (Watts et al, 2021).…”

Section: Issues With Failing To Recognize Heterogeneity In Item Setsmentioning

confidence: 99%

Focusing Narrowly on Model Fit in Factor Analysis can Mask Construct Heterogeneity and Model Misspecification: Applied Demonstrations Across Sample and Assessment Types

Stanton¹,

Watts²,

Levin‐Aspenson³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

This study builds upon recent research indicating that focusing narrowly on model fit when evaluating factor analytic models can lead to problematic inferences regarding the nature of item sets, as well as how models should be applied to inform measure development and validation. Specifically, we demonstrate that an overreliance on model fit may lead to (a) incorrect conclusions that heterogeneous item sets reflect narrower homogeneous constructs and (b) the retention of potentially problematic items when developing assessment measures. We use both interview data from adult outpatients (N = 2,149) and self-report data from adults recruited online (N = 547) to demonstrate the importance of these issues across sample types and assessment methods. Following demonstrations with these data, we make recommendations focusing on how theory and other model characteristics (e.g., factor loading patterns) should be considered in addition to information provided by model fit indices when evaluating factor analytic models.

show abstract

“…Researchers may mistakenly interpret heterogeneous sets of items as being homogeneous in nature if model fit is "good" or "acceptable" for single-factor models (Watts et al, 2021).…”

Section: Issues With Failing To Recognize Heterogeneity In Item Setsmentioning

confidence: 99%

Focusing Narrowly on Model Fit in Factor Analysis can Mask Construct Heterogeneity and Model Misspecification: Applied Demonstrations Across Sample and Assessment Types

Stanton¹,

Watts²,

Levin‐Aspenson³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…In recent years, there have been substantial developments in the measurement of psychopathological phenomena, including addiction and BAs. Still, there is room for considerable improvement [137][138][139][140].…”

Section: Implications For Research and Interventionsmentioning

confidence: 99%

Subpopulations of Addictive Behaviors in Different Sample Types and Their Relationships with Gender, Personality, and Well-Being: Latent Profile vs. Latent Class Analysis

Atroszko

Charzyńska

2021

IJERPH

View full text Add to dashboard Cite

Background: Relatively strong theoretical assumptions and previous studies concerning co-occurring addictive behaviors suggest a subpopulation representing general proclivity to behavioral addictions (BAs), and there are gender-specific subpopulations. This study aimed to compare latent profile analysis (LPA) and latent class analysis (LCA) as the methods of investigating different clusters of BAs in the general student population and among students positively screened for at least one BA. Participants and procedure: Analyses of six BAs (study, shopping, gaming, Facebook, pornography, and food) and their potential antecedents (personality) and consequences (well-being) were conducted on a full sample of Polish undergraduate students (N = 1182) and a subsample (n = 327) of students including individuals fulfilling cutoff for at least one BA. Results: LPA on the subsample mostly replicated the previous four profiles found in the full sample. However, LCA on a full sample did not replicate previous findings using LPA and showed only two classes: those with relatively high probabilities on all BAs and low probabilities. LCA on the subsample conflated profiles identified with LPA and classes found with LCA in the full sample. Conclusions: LCA on dichotomized scores (screened positively vs. negatively) were less effective in identifying clear patterns of interrelationships between BAs based on relatively strong theoretical assumptions and found in previous research. BAs can be investigated on the whole spectrum of behavior, and person-centered analyses might be more useful when they are based on continuous scores. This paper provides more detailed analyses of the four basic clusters of BAs, prevalence, and co-occurrence of particular BAs within and between them, their gender and personality risk factors, relationships to well-being, and their interrelationships as emerging from the results of this and previous studies.

show abstract

“…Next, we conducted a series of EFAs to probe evidence of anomalous results in previous work and to test whether the models derived by CFA would also emerge with limited input from the researcher (Watts, Boness, et al, 2020). Confirmatory approaches are clearly valuable given a strong theory about the latent structure of a set of observed variables, but contemporary CFA model comparisons typically proceed in a covertly exploratory manner (Asparouhov & Muthén, 2009;Brown, 2015;Browne, 2001;MacCallum et al, 1992;Schmitt et al, 2018).…”

Section: Combining Exploratory and Confirmatory Analysesmentioning

confidence: 99%

Misbegotten Methodologies and Forgotten Lessons from Tom Swift’s Electric Factor Analysis Machine: A Demonstration with Competing Structural Models of Psychopathology

Greene¹,

Al²,

Mk³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Confirmatory factor analysis (CFA) and its bifactor models are popular in empirical investigations of the factor structure of psychological constructs. CFA offers straightforward hypothesis testing but has notable pitfalls, such as the imposition of strict assumptions (i.e., simple structure) that obscure unmodeled complexity. Due to the limitations of bifactor CFAs, they have yielded anomalous results across samples and studies that suggest model misspecification (e.g., evaporating specific factors and unexpected loadings). We propose the use of exploratory factor analysis (EFA) to evaluate the structural validity of CFA solutions—either before or after the estimation of more restrictive CFA models—to (1) identify model misspecifications that may drive anomalous estimates and (2) confirm CFA models by examining whether hypothesized structures emerge with limited researcher input. We evaluated the degree to which predominant factor structures were invariant across contexts along the exploratory-confirmatory continuum and demonstrate how poor methodological choices can distort results and impede theory development. All models fit well, but there were numerous differences in replicability and substantive interpretability. Several similarities emerged between bifactor CFA and EFA models, including evidence of overextraction, the collapse of specific factors onto the general factor, and subsequent shifts in how the general factor was defined. We situate these methodological shortcomings within the broader literature on structural models of psychopathology, articulate implications for theories (such as the p-factor) that are borne out of factor analysis, outline several remedies for problems encountered when performing exploratory bifactor analysis, and propose alternative specifications for confirmatory bifactor models.

show abstract

Does crude measurement contribute to observed unidimensionality of psychological constructs? An example with DSM-5 alcohol use disorder

Cited by 14 publications

References 50 publications

Focusing Narrowly on Model Fit in Factor Analysis can Mask Construct Heterogeneity and Model Misspecification: Applied Demonstrations Across Sample and Assessment Types

Focusing Narrowly on Model Fit in Factor Analysis can Mask Construct Heterogeneity and Model Misspecification: Applied Demonstrations Across Sample and Assessment Types

Subpopulations of Addictive Behaviors in Different Sample Types and Their Relationships with Gender, Personality, and Well-Being: Latent Profile vs. Latent Class Analysis

Misbegotten Methodologies and Forgotten Lessons from Tom Swift’s Electric Factor Analysis Machine: A Demonstration with Competing Structural Models of Psychopathology

Contact Info

Product

Resources

About