RMSEA, CFI, and TLI in structural equation modeling with ordered categorical data: The story they tell depends on the estimation methods

Abstract

In structural equation modeling, application of the root mean square error of approximation (RMSEA), comparative fit index (CFI), and Tucker-Lewis index (TLI) highly relies on the conventional cutoff values developed under normal-theory maximum likelihood (ML) with continuous data. For ordered categorical data, unweighted least squares (ULS) and diagonally weighted least squares (DWLS) based on polychoric correlation matrices have been recommended in previous studies. Although no clear suggestions exist regarding the application of these fit indices when analyzing ordered categorical variables, practitioners are still tempted to adopt the conventional cutoff rules. The purpose of our research was to answer the question: Given a population polychoric correlation matrix and a hypothesized model, if ML results in a specific RMSEA value (e.g., .08), what is the RMSEA value when ULS or DWLS is applied? CFI and TLI were investigated in the same fashion. Both simulated and empirical polychoric correlation matrices with various degrees of model misspecification were employed to address the above question. The results showed that DWLS and ULS lead to smaller RMSEA and larger CFI and TLI values than does ML for all manipulated conditions, regardless of whether or not the indices are scaled. Applying the conventional cutoffs to DWLS and ULS, therefore, has a pronounced tendency not to discover model-data misfit. Discussions regarding the use of RMSEA, CFI, and TLI for ordered categorical data are given.

Assessing mental well-being in a Sinhala speaking Sri Lankan population: validation of the WHO-5 well-being index

“…When tested in a post-hoc model covarying the two items, a non-significant Chi-square (p > 0.05) and better fit indices with RMSEA < 0.05 were obtained. Xia and Yang state that further studies are needed to seek alternative methods for goodness-of-fit evaluation with ordered categorical data [35]. Shi et al has demonstrated that SRMR is relatively free from the choice of estimation method and the same population cut-offs can be applied regardless of the estimation method employed [34].…”

“…Four other approximate fit indices were considered: RMSEA < 0.050 and < 0.080 for close and reasonable fit, respectively: Comparative fit index (CFI) and Tucker-Lewis Index (TLI) of > 0.900 and > 0.950 were considered for acceptable and excellent fit, respectively [32]; Standard Root Mean Square Residual (SRMR) of < 0.05 was also considered because this index is independent of the estimator used [33]. Since we used the DWLS estimator for our analysis, SRMR has been recommended as the main index to test model fit [34][35][36][37].…”

COVID-19 Induced Anxiety and Protective Behaviors During COVID-19 Outbreak: Scale Development and Validation

“…The 4 specific criteria are (1) CFI>0.9; (2) RMSEA<0.09; (3) SRMR<0.09; (4) chi-square/df <5. [31][32][33] The internal consistency was fair (Cronbach's alpha=0.78). For the PBCS, we assigned 5 items into the factor of routine protective behaviors (RPB), 6 items into post-exposure protective behaviors (PPB), and 3 items into post-exposure risky behaviors (PRB).…”

Measuring and managing brand loyalty of banks` clients

“…They are the degrees of freedom (CMIN/df), comparative fit index (CFI), goodness of fit index (GFI), the Tucker-Lewis index (TLI) and root mean square error of approximation (RMSEA) (Kumar, 2019). CFI, GFI and TLI are indices that measure the incremental fit of the model compared to the fit of the hypothesized model to the fit of the baseline model (a base-line model is a model with the worst fit (Xia & Yang, 2019). TLI is a non-normed index (NNFI) as it is also sometimes referred to (Kumar, 2019).…”