Reliability, and Convergent and Discriminant Validity of Gaming Disorder Scales: A Meta-Analysis

Abstract

Background: An association between gaming disorder (GD) and the symptoms of common mental disorders is unraveled yet. In this preregistered study, we quantitatively synthesized reliability, convergent and discriminant validity of GD scales to examine association between GD and other constructs.Methods: Five representative GD instruments (GAS-7, AICA, IGDT-10, Lemmens IGD-9, and IGDS9-SF) were chosen based on recommendations by the previous systematic review study to conduct correlation meta-analyses and reliability generalization. A systematic literature search was conducted through Pubmed, Proquest, Embase, and Google Scholar to identify studies that reported information on either reliability or correlation with related variables. 2,124 studies were full-text assessed as of October 2020, and 184 were quantitatively synthesized. Conventional Hedges two-level meta-analytic method was utilized.Results: The result of reliability generalization reported a mean coefficient alpha of 0.86 (95% CI = 0.85–0.87) and a mean test-retest estimate of 0.86 (95% CI = 0.81–0.89). Estimated effect sizes of correlation between GD and the variables were as follows: 0.33 with depression (k = 45; number of effect sizes), 0.29 with anxiety (k = 37), 0.30 with aggression (k = 19), –0.22 with quality of life (k = 18), 0.29 with loneliness (k = 18), 0.56 with internet addiction (k = 20), and 0.40 with game playtime (k = 53), respectively. The result of moderator analyses, funnel and forest plots, and publication bias analyses were also presented.Discussion and Conclusion: All five GD instruments have good internal consistency and test-retest reliability. Relatively few studies reported the test-retest reliability. The result of correlation meta-analysis revealed that GD scores were only moderately associated with game playtime. Common psychological problems such as depression and anxiety were found to have a slightly smaller association with GD than the gaming behavior. GD scores were strongly correlated with internet addiction. Further studies should adopt a rigorous methodological procedure to unravel the bidirectional relationship between GD and other psychopathologies.Limitations: The current study did not include gray literature. The representativeness of the five tools included in the current study could be questioned. High heterogeneity is another limitation of the study.Systematic Review Registration: [https://www.crd.york.ac.uk/PROSPERO/], identifier [CRD42020219781].

Reliability Generalization Meta-Analysis of Internet Gaming Disorder Scale

“…Finally, in comparison with another meta-analysis analyzing the reliability of various instruments [ 31 ], the two polytomous versions of the IGDS (IGDS27P and IGDS9P) show higher reliability scores than those found on other similar instruments. This highlights that these versions may be more suitable for the assessment of the IGD.…”

“…In our study, the four original versions [ 9 ] are analyzed, including the 9-item and 27-item versions with polytomous and dichotomous response. The election of this instrument by Yoon et al [ 31 ] is based on the results of the systematic review by King et al [ 103 ], in which only the 9-item dichotomous and 27-item dichotomous and polytomous versions [ 103 ] were included (p. 4, Table 1 ). Therefore, not considering the four versions could mean an error in the search, since our sample (made in 2021 and updated in June 2022, consulting Scopus, WoS, PsycInfo, and Dialnet databases) already contains articles that use the not-included version (when the review was conducted in these and other databases by King et al [ 103 ]).…”

“…This fact can be both a strength, because it allows greater generalization of results by having a larger sample, and a weakness if it implies a reduction in quality. In the study by Yoon et al [ 31 ], the quality criterion used was that the journal of publication had peer review, and therefore, we reviewed our gray literature and concluded that it also met this criterion. All the articles that formed part of the gray literature were published in peer-reviewed journals, thus providing a larger sample and more generalizable results.…”

“…After a previous search, no examples of this type of meta-analysis have been found for the IGDS. However, while it was being carried out, a study was detected that included a reliability generalization meta-analysis of the IGDS [ 31 ], with which there are substantial differences in the included versions of the IGDS, the meta-analytic models employed, the sample size, and the sample size, and the assessment of heterogeneity. Therefore, this study includes the four main versions of the IGDS, as well as more models and reliability and heterogeneity estimators, and discusses which of the four versions may be more reliable.…”

Editorial: Internet Addiction & Gaming Disorders in Children and Adolescents

“…A meta-analysis that studied the reliability and validity of 5 Gaming Disorder Scales (GAS-7, AICA, IGDT-10, Lemmens IGD-9, and IGDS9-SF) showed that they had good internal consistency and test-retest reliability. However, few studies had checked for test-retest reliability ( 9 ). A China-based study evaluated the reliability and internal consistency of the “Chinese Internet Gaming Disorder Checklist (C-IGDC)” in determining the presence of Internet gaming disorder ( Chen et al ).…”