Detecting Misspecified Multilevel Structural Equation Models with Common Fit Indices: A Monte Carlo Study

Hsu, Hsien-Yuan; Kwok, Oi‐Man; Lin, Jr Huang; Acosta, Sandra

doi:10.1080/00273171.2014.977429

Cited by 90 publications

(116 citation statements)

References 41 publications

Supporting

Mentioning

103

Contrasting

Order By: Relevance

“…Hsu et al (2015) found ICC had no substantial impact on the effectiveness of traditional fit indices. The major reason that Hsu et al (2015) did not find ICC as an important factor is that the traditional single-level fit indices were the main focus and the performance of these traditional fit indices was in general overpowered by the within-level model misspecification.…”

Section: The Role Of Icc In the Performance Of Traditional And Level-mentioning

confidence: 91%

“…Our study extended this line of research specifically by furthering Hsu et al's (2015) study with the focus on how the effectiveness of ''level-specific'' fit indices is affected by varying ICC values. We aim to provide better understanding of the utilization of level-specific fit indices in MSEM given data with different levels of ICC, which has not yet been thoroughly investigated in previous level-specific fit index studies.…”

Section: Introductionmentioning

confidence: 95%

See 1 more Smart Citation

The Impact of Intraclass Correlation on the Effectiveness of Level-Specific Fit Indices in Multilevel Structural Equation Modeling

Hsu

Lin

Kwok

et al. 2016

Educational and Psychological Measurement

Self Cite

View full text Add to dashboard Cite

Several researchers have recommended that level-specific fit indices should be applied to detect the lack of model fit at any level in multilevel structural equation models. Although we concur with their view, we note that these studies did not sufficiently consider the impact of intraclass correlation (ICC) on the performance of level-specific fit indices. Our study proposed to fill this gap in the methodological literature. A Monte Carlo study was conducted to investigate the performance of (a) level-specific fit indices derived by a partially saturated model method (e.g., [Formula: see text] and [Formula: see text]) and (b) [Formula: see text] and [Formula: see text] in terms of their performance in multilevel structural equation models across varying ICCs. The design factors included intraclass correlation (ICC: ICC1 = 0.091 to ICC6 = 0.500), numbers of groups in between-level models (NG: 50, 100, 200, and 1,000), group size (GS: 30, 50, and 100), and type of misspecification (no misspecification, between-level misspecification, and within-level misspecification). Our simulation findings raise a concern regarding the performance of between-level-specific partial saturated fit indices in low ICC conditions: the performances of both [Formula: see text] and [Formula: see text] were more influenced by ICC compared with [Formula: see text] and . However, when traditional cutoff values (≤ 0.06; ≥ 0.95;≤ 0.08) were applied, [Formula: see text] and [Formula: see text] were still able to detect misspecified between-level models even when ICC was as low as 0.091 (ICC1). On the other hand, both [Formula: see text] and [Formula: see text] were not recommended under low ICC conditions.

show abstract

Section: The Role Of Icc In the Performance Of Traditional And Level-mentioning

confidence: 91%

Section: Introductionmentioning

confidence: 95%

The Impact of Intraclass Correlation on the Effectiveness of Level-Specific Fit Indices in Multilevel Structural Equation Modeling

Hsu

Lin

Kwok

et al. 2016

Educational and Psychological Measurement

Self Cite

View full text Add to dashboard Cite

show abstract

“…We evaluated the models based on the Comparative Fit Index (CFI; Bentler, 1990), root mean square error of approximation (RMSEA; Steiger & Lind, 1980), the standardized root mean square residual for the within-level (i.e., individual-level) model (SRMR-within; Hu & Bentler, 1999), and the standardized root mean square residual for the between-level (i.e., team-level) model (SRMR-between;Hsu, Kwok, Lin, & Acosta, 2015). The model with an adequate fit should have CFI greater than 0.90, RMSEA less than 0.06, SRMR-within less than 0.08 (Hu & Bentler, 1999), and SRMR-between less than 0.14 (Hsu et al, 2015).…”

Section: Data Analysesmentioning

confidence: 99%

How to build your team for innovation? A cross‐level mediation model of team personality, team climate for innovation, creativity, and job crafting

Jiang

Wang

2019

J Occupat & Organ Psyc

View full text Add to dashboard Cite

Although multiple theories propose that team proactive personality shapes team climate for innovation, this theoretical proposition is yet to be tested by empirical studies. Drawing upon multiple theories (e.g., sense‐making theory and theories regarding social norms), we hypothesized a cross‐level mediation model of the effects of different operationalizations of team proactive personality on team climate for innovation, as well as individual creative performance and job crafting. We tested our hypothesized model using a 2‐wave longitudinal design collecting data from a sample of 187 employees and their supervisors nested within 38 teams from an Information Technology company. The results of multilevel mediation path analysis indicated that team proactive personality (i.e., mean, minimum, and maximum) was significantly and positively related to team climate for innovation. Further, team proactive personality at Time 1 exerted a cross‐level influence on individual creative performance (rated by supervisors) and job crafting at Time 2, through team climate for innovation. Finally, different operationalizations (mean, maximum, minimum, and standard deviation) of team proactive personality revealed differential relationships with team climate for innovation. These findings suggest that team proactive personality can influence outcomes at different levels of analyses. Theoretical and practical implications of these findings are discussed. Practitioner points Team proactive personality (i.e., mean, maximum, and minimum) positively related to team climate for innovation. Team climate for innovation positively related to both individual creativity and job crafting. Team climate for innovation mediated the top‐down effects of team proactive personality on individual creativity and job crafting. Employers may consider proactive personality as one criterion at the stages of selecting and staffing of team members for creative jobs and creative team projects.

show abstract

“…One common approach to model evaluation uses fit indices (e.g., the root mean square error of approximation [RMSEA], comparative fit index [CFI], Tucker-Lewis index [TLI], and standardized root mean square residual [SRMR]) to assess the model fit. However, because a traditional MLGCM comprises both between and within models, it has been suggested that the models at different levels should be evaluated separately by level-specific fit indices (Hox, 2010;Hsu, Kwok, Acosta, & Lin, 2015;Ryu, 2014;Ryu & West, 2009). Studies contributing to understanding the performance of level-specific fit indices in MSEM have been conducted in the context of multilevel confirmatory factor analysis (MCFA; e.g., Hsu, Lin, Kwok, Acosta, & Willson, 2016;Ryu & West, 2009), multilevel path models (Ryu, 2014), and multilevel nonlinear models (Schermelleh-Engel, Kerwer, & Klein, 2014).…”

mentioning

confidence: 99%

Evaluating fit indices in a multilevel latent growth curve model: A Monte Carlo study

et al. 2018

Self Cite

View full text Add to dashboard Cite

The multilevel latent growth curve model (MLGCM), which is subsumed by the multilevel structural equation modeling framework, has been advocated as a means of investigating individual and cluster trajectories. Still, how to evaluate the goodness of fit of MLGCMs has not been well addressed. The purpose of this study was to conduct a systematic Monte Carlo simulation to carefully investigate the effectiveness of (a) level-specific fit indices and (b) target-specific fit indices in an MLGCM, in terms of their independence from the sample size's influence and their sensitivity to misspecification in the MLGCM that occurs in either the between-covariance, betweenmean, or within-covariance structure. The design factors included the number of clusters, the cluster size, and the model specification. We used Mplus 7.4 to generate simulated replications and estimate each of the models. We appropriately controlled the severity of misspecification when we generated the simulated replications. The simulation results suggested that applying RMSEA T_S_COV , TLI T _ S _ COV , and SRMR B maximizes the capacity to detect misspecifications in the between-covariance structure. Moreover, the use of RMSEA PS _ B , CFI PS _ B , and TLI PS _ B is recommended for evaluating the fit of the between-mean structure. Finally, we found that evaluation of the within-covariance structure turned out to be unexpectedly challenging, because none of the within-level-specific fit indices (RMSEA PS _ W , CFI PS _ W , TLI PS _ W , and SRMR W) had a practically significant sensitivity. Keywords Fit index. Model evaluation. Multilevel latent growth curve model. Multilevel structural equation modeling A panel study is a powerful longitudinal design in which data are observed or gathered from exactly the same people, group, or organization across multiple time points (Neuman, 2009). Panel studies allow researchers to investigate a moving picture of observed units over time (i.e., a trajectory), rather than a single snapshot, as in cross-sectional studies. During the past few decades, two-stage cluster sampling (TCS) has been widely adopted for most large-scale panel studies (e.g., the Education Longitudinal Study of 2002, Ingels et al.

show abstract

Detecting Misspecified Multilevel Structural Equation Models with Common Fit Indices: A Monte Carlo Study

Cited by 90 publications

References 41 publications

The Impact of Intraclass Correlation on the Effectiveness of Level-Specific Fit Indices in Multilevel Structural Equation Modeling

The Impact of Intraclass Correlation on the Effectiveness of Level-Specific Fit Indices in Multilevel Structural Equation Modeling

How to build your team for innovation? A cross‐level mediation model of team personality, team climate for innovation, creativity, and job crafting

Evaluating fit indices in a multilevel latent growth curve model: A Monte Carlo study

Contact Info

Product

Resources

About