Measuring Internet Gaming Disorder and Gaming Disorder: A Qualitative Content Validity Analysis of Validated Scales

Abstract

Numerous instruments have been developed to measure gaming-related health problems based on “internet gaming disorder” (IGD) in the third section of the Diagnostic and Statistical Manual of Mental Disorders (5th ed.) and “gaming disorder” (GD) in the International Classification of Diseases (11th rev.). However, the criteria in the manuals tend to be operationalized in numerous diverse ways, which can make screening outcomes incomparable. A content validity analysis is needed to reassess the relationships between the diagnostic criteria and the items that operationalize them. The IGD and GD criteria were divided into sematic components. A qualitative content validity analysis was carried out for all items employed by the 17 instruments that claim to measure either construct by their criteria in English. In all but one instrument, the operationalizations did not include all criterion components. There were two main reasons found for this: the components had simply been left out or had been alternatively modified into other components. Criteria that were vaguely described in the manuals were sources of lower content validity items. The study implies that many of the problems in IGD and GD measurement derive from criteria operationalization and original manual descriptions. The conclusion provides practical recommendations that researchers can apply to improve the content validity of their measurement.

Fuzzy constructs in technology usage scales

“…Hence, measures based on this model may conceptually overlap. However, technology addiction scales can have different ontological groundings (e.g., pathological gambling vs components model of addiction), which likely contributes to variances in prevalence rates (Karhulahti, Martončik, & Adamkovič, 2021). Despite the lack of clarity regarding definitions, evidence, and measurement, gaming disorder remains a part of the ICD-11 (Karhulahti, Martončik, & Adamkovič, 2021).…”

“…However, technology addiction scales can have different ontological groundings (e.g., pathological gambling vs components model of addiction), which likely contributes to variances in prevalence rates (Karhulahti, Martončik, & Adamkovič, 2021). Despite the lack of clarity regarding definitions, evidence, and measurement, gaming disorder remains a part of the ICD-11 (Karhulahti, Martončik, & Adamkovič, 2021). Even at the level of face validity, these inconsistencies serve to undermine how scales could diagnose any condition (Karhulahti, Vahlo, Munukka, Koskimaa, & von Bonsdorff, 2021).…”

Network Structures of Internet Gaming Disorder and Gaming Disorder: Symptom Operationalization Causes Variation

“…ICD-11 based GD was measured using the English version of the Gaming Disorder Test (Pontes et al, 2019). For RQ2, we selected alternative symptom operationalizations with high content validity, based on a previous semantic analysis (Karhulahti, et al, 2021). Items measuring withdrawal, loss of interests, and continued use were selected from C-IGDS (Sigerson et al, 2017); tolerance from PIE-9 (Pearcy et al, 2016).…”

“…In theory, after the DSM-5 (2013) and ICD-11 (2019) were published, new screening scales should have a shared semiology. However, content validity analysis shows that scales following the diagnostic manuals operationalize their symptoms in substantially different ways-thus measuring potentially different constructs (Karhulahti et al, 2021). Whether these different operationalizations result in different screening outcomes remains an open empirical question, and our network approach is a fitting method for seeking answers to it.…”

“…One of the foremost challenges in the field is that both DSM-5 and ICD-11 based IGD/GD screening instruments operationalize the official diagnostic criteria in many different ways (Karhulahti et al, 2021). This means that results produced by one instrument can differ significantly from those of another instrument-even when both claim to measure the same symptoms.…”