Consequences of Not Conducting Measurement Invariance Tests in Cross-Cultural Studies: A Review of Current Research Practices and Recommendations

Abstract

The Problem Cross-cultural research has received substantial attention from both academia and practice as it contributes to expand current theory and implements culturally successful human resource strategies. Although the quantity of this type of research has increased, several researchers have raised methodological concerns that the majority of cross-cultural research has simply assumed or ignored measurement invariance. The Solution In this article, we first provide the meaning for measurement invariance, discuss why it is important, and then explain stepwise confirmatory factor analysis procedures to test measurement invariance. We also diagnose the current research practice in the field of human resource development (HRD) based on a review of cross-cultural, comparative research published in the major HRD journals. Finally, we demonstrate that the group difference test results that have been found without ensuring measurement invariance can, in fact, be false. The Stakeholders This article contributes to the HRD literature and practice in two ways. First, HRD researchers are invited to recognize the importance of sophisticated research methodology, such as measurement invariance, and to examine item bias across different groups so they can make a meaningful and valid comparison. The same attention is advisable to any practitioner who attempts to identify group differences using multinational/cultural data. This article also provides HRD scholars and practitioners with specific multigroup confirmatory factor analysis (MGCFA) procedures to facilitate empirical tests of measurement models across different groups and thus disseminate the methodological advances in the field of HRD. It is our hope that the present article raises awareness, circulates relevant knowledge, and encourages more HRD scholars to think critically about measurement.

The Necessity of Testing Measurement Invariance in Cross-Cultural Research: Potential Bias in Cross-Cultural Comparisons With Individualism– Collectivism Self-Report Scales

“…This finding convincingly shows that establishing scalar invariance is an important step in the analysis of this type of cross-cultural data before any actual comparison of mean scores because it improves the credibility of the results and their interpretation. Our results agree with other previous studies conducted in related research fields, which used both real and simulated data and which illustrated that ignoring measurement (non-)invariance might lead to biased results (Chen, 2008; Guenole & Brown, 2014; Hsiao & Lai, 2018; Jeong & Lee, 2019; Oberski, 2014; Schmitt et al, 2011; Steinmetz, 2013; Widaman & Reise, 1997).…”

The Necessity of Testing Measurement Invariance in Cross-Cultural Research: Potential Bias in Cross-Cultural Comparisons with Individualism–Collectivism Self-Report Scales

“…For these reasons, we agree with scholars on the call to abandon the "traditional" approach and emphasize the importance of implementing measurement invariance testing in cross-cultural research in general (e.g., Boer et al, 2018;Chen, 2008;Jeong & Lee, 2019;Fischer & Karl, 2019; as well as in self-report I/C research (e.g., Chen & West, 2008;Cozma, 2011;Lacko et al, under review;Levine et al, 2003a;2003b;Oyserman et al, 2002;Schimmack et al, 2005). Incorporating measurement invariance testing in I/C research therefore has a great potential to increase the validity and reliability of crosscultural comparisons with self-report scales and to provide the necessary fundamentals for an examination of the real differences in levels of psychological phenomena across cultures.…”

Exploring consumers’ attitude formation toward their own brands when in crisis: cross-national comparisons between USA and China

“…In summary, all the constructs from both country samples satisfied all the required tests for reliability and validity. Accessing measurement invariance of the measured constructs By nature, cross-cultural studies adopt measurement instruments that cannot be universally applied to multiple cultures and/or nations because the instruments developed in one culture might possibly cause an unexpected bias toward another culture; the required linguistic translation process might mislead subjects' opinions on survey questions (Burton, 2015;Jeong and Lee, 2019). Accordingly, to avoid obtaining misleading results due to invalid measurement tools, all crosscultural studies that quantitatively compare groups of people from different cultures should conduct invariance tests that enable researchers to measure and confirm whether the instrument is operative and applicable across different cultures, although a majority of the cultural studies in academia, unfortunately, disregard this statistical consideration.…”