Test-Retest Measurement Invariance of the Nine-Item Internet Gaming Disorder Scale in Two Countries: A Preliminary Longitudinal Study

Abstract

The reliable longitudinal assessment of Internet Gaming Disorder (IGD) behaviors is viewed by many as a pivotal clinical and research priority. The present study is the first to examine the test-retest measurement invariance of IGD ratings, as assessed using the short-form nine-item Internet Gaming Disorder Scale (IGDS9-SF) over an approximate period of 3 months, across two normative national samples. Differences referring to the mode of the data collection (faceto-face [FtF] vs. online) were also considered. Two sequences of successive multiple group confirmatory factor analyses (CFAs) were calculated to longitudinally assess the psychometric properties of the IGDS9-SF using emergent adults, gamers from (i) the United States of America (USA; N = 120, 18-29 years, Mean age = 22.35, 51.6% male) assessed online and; and (ii) Australia (N = 61, 18-31 years, Mean age = 23.02, 75.4% male) assessed FtF. Configural invariance was established across both samples, and metric and scalar invariances were supported for the USA sample. Interestingly, only partial metric (factor loadings for Items 2 and 3 non-invariant) and partial scalar invariance (i.e., all thresholds of Items 1 and 2, and thresholds 1, 3, for Items 4, 6, 8, and 9 non-invariant) were established for the Australian sample. Findings are discussed in the light of using IGDS9-SF to assess and monitor IGD behaviors over time in both in clinical and non-clinical settings.

Editorial: Internet Gaming Disorder: A Pathway Towards Assessment Consensus

“…However, the relative agreement in the definition of the construct that has been achieved, which constitutes a necessary requirement for the valid and reliable assessment of disordered gaming behaviors, is not sufficient (Stavropoulos et al, 2019a,b,c). Adequate psychometric properties of the scales utilized, to assess the disordered gaming classifications officially defined, are required for the accurate estimation and the cross-countries comparability of the syndrome's prevalence and incidence rates (Gomez et al, 2018).…”

“…Interestingly, and despite the ongoing, often chaotic and confusing, debate around the disordered gaming construct, the need of robust measures to psychometrically measure it has been stressed (Stavropoulos et al, 2018). In that line, significant progress has been made in regards to defining, understanding and confirming: (a) The dimensional-structure of the behavior; (b) How different criteria and scores translate (metric and scalar invariance) across populations; (c) Differential diagnostic criterion functioning (through the use of item response theory) and; (d) the psychometric stability of disordered gaming measurement over time (Kuss et al, 2017; De Palo et al, 2018; Gomez et al, 2018; Pontes et al, 2019; Stavropoulos et al, 2019c).…”

“…However, challenges in the field of disordered gaming assessment still remain. Scholars continue to disagree about the nature of the behavior (Kardefelt-Winther et al, 2017), different instruments hindering international comparability are still employed (Costa and Kuss), while the number of measurement invariance studies, targeting in particular issues of scalar invariance (whether the same scores indicate the same severity) across populations of different genders, cultures, and developmental stages (although increasing) are rare (Stavropoulos et al, 2018, 2019c). The application of modern psychometric methodologies such as network analysis, that would illustrate the nature of the associations between the different criteria, is absent; while there is concurrently a dearth of Item Response Theory invariance studies to better highlight the potential different diagnostic functioning of certain criteria across different populations (Gomez et al, 2018).…”

Body Appreciation Scale (BAS-2): Measurement Invariance across Genders and Item Response Theory Examination.

“…Considering the stringent nature of χ 2 comparisons between the con gural, the metric and the scalar levels (Δ χ 2 < .05), we also evaluated incremental t differences in CFI and RMSEA values (ΔCFI > .010, ΔRMSEA > .015, [12,18]). It should be noted that if full measurement invariance is not attained when comparing models, partial invariance can be explored to determine the source of noninvariance by sequentially freeing constrained parameters across different items, until non-signi cant difference across models is achieved [21].…”

“…Establishing MI across observed groups, such as genders, is of paramount importance to claim signi cance over inferred comparative observations [21]. MI can be considered as a test of heterogeneity, that evaluates whether the measurement properties of a construct remain stable across groups, thus securing meaningful comparisons.…”