Development and Validation of an Item Bank for Drug Dependence Measurement Using Computer Adaptive Testing

Liu, Siyang; Tu, Dongbo; Cai, Yan

doi:10.1080/10826084.2020.1801743

Cited by 3 publications

(12 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This indicates that there is a deviation in the item when the variation in R 2 is greater than .02; then, it should be deleted (Choi et al, 2011). Referring to existing IRT-related and CAT-related studies (Gaynes et al, 2002; Liu et al, 2020; Tan et al, 2018), we selected two demographic variables, which have been the focus of DIF detection in these studies: region (rural, city) and gender (male, female). In this study, we conducted DIF analysis using the R package lordif (Version 0.3-3; Choi et al, 2011).…”

Section: Discussionmentioning

confidence: 99%

Developing an Item Bank of Computerized Adaptive Testing for Eating Disorders in Chinese University Students

Liu

Zhang

et al. 2022

Sage Open

Self Cite

View full text Add to dashboard Cite

We aimed to develop an item bank of computerized adaptive testing for eating disorders (CAT-ED) in Chinese university students to increase measurement precision and improve test efficiency. A total of 1,025 Chinese undergraduate respondents answered a series of questions about eating disorders in a paper-pencil test. A total of 133 items from four well-validated Chinese-version scales of eating disorders were used to construct the item bank of CAT-ED with the following analysis. First, unidimensionality, model fit, local independence, item ﬁt, discrimination and differential item functioning (DIF) were tested. Then, two simulation studies were applied to test the CAT-ED’s effectivity and rationality by calculating concurrent criterion-related validity, sensitivity and speciﬁcity. The ﬁnal item bank comprised 77 items, which met the requirements of local independence, item ﬁt, high discrimination and no differential item functioning in CAT. The mean number of administered items in CAT with the stopping rule fixed at SE ≤ 0.3 was 11 items. The obtained results showed that CAT-ED had acceptable reliability, validity, sensitivity and speciﬁcity.

show abstract

Section: Discussionmentioning

confidence: 99%

Developing an Item Bank of Computerized Adaptive Testing for Eating Disorders in Chinese University Students

Liu

Zhang

et al. 2022

Sage Open

Self Cite

View full text Add to dashboard Cite

show abstract

“…Smits et al (2011) and Yudien et al (2019) selected the first item that is most informative for the average latent trait score. In this study, we followed the approach of Liu et al (2020) and Magis and Barrada (2017) by randomly selecting the first item from the full DSQ for each participant.…”

Section: Methodsmentioning

confidence: 99%

“…In this study, the CATs used a stopping rule that combined measurement precision and an upper limit of administered items (Butler et al, 2017; Flens et al, 2022; Liu et al, 2020; Smits et al, 2011; Yudien et al, 2019). Specifically, the CAT measurement was terminated when it reached a predefined SE of theta or the maximum length of administered items.…”

Section: Methodsmentioning

confidence: 99%

“…Of these, the 40-item DSQ is a widely used tool for assessing the severity of amphetamine addiction and monitoring treatment outcomes and has been established as a reputable instrument in both research and clinical settings (Dickerson et al, 2012; Galloway et al, 2008; Hassani-Abharian et al, 2016; Rathitharan et al, 2021). However, the DSQ was developed based on the classical test theory (CTT), resulting in summed scores and a requirement that respondents complete the entire questionnaire to ensure comparability, leading to greater burden and potentially lower response quality (Liu et al, 2020). In fact, as mentioned earlier, time is limited for care providers and patients in the clinical setting, and assessments should be brief yet reliable for all patients to minimize costs (Flens et al, 2016), which helps extend the use of assessment tool to nonspecialty health care (Stewart et al, 2022).…”

mentioning

confidence: 99%

“…Butler et al (2017) suggested a CAT version of the Addiction Severity Index–Multimedia Version (ASI-MV) for assessing the extent of substance use problem. Recently, Liu et al (2020) established an item bank for the measurement of drug dependence through CAT, whereas Gibbons et al (2020) improved substance use disorder (SUD) screening and measurement with the application of CAT. Furthermore, Hulvershorn et al (2022) developed a screening, measurement, and diagnostic tool (CAT-SUD-E) for SUD assessment through CAT.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Applying computerized adaptive testing to the Desires for Speed Questionnaire in the Chinese population: A simulation study.

Liu

Guo

et al. 2023

Psychological Assessment

Self Cite

View full text Add to dashboard Cite

The current methods for measuring patient-reported outcomes for amphetamine (speed) craving have limitation ability to adapt to the needs of individual patients while maintaining consistency in their scores. This study aimed to investigate whether the 40-item Desires for Speed Questionnaire (DSQ) could be improved for assessing clinical subjects using computerized adaptive testing (CAT). A sample of 677 participants from four drug addiction treatment centers in China was utilized in the study. Two types of analysis were conducted using the response data. First, the psychometric properties of all items were evaluated to meet the requirements of CAT. Second, multiple CAT simulations were carried out using real response data. The results indicated that the CAT method, which only required a small number of items (50%-75%), produced results that were only slightly different from the full DSQ assessment in terms of measuring amphetamine craving and criterion validity. In conclusion, this study suggests that developing a DSQ CAT for clinical subjects is useful as it leads to more efficient measurement without compromising the reliability of the test outcomes. Public Significance StatementThis study highlights the potential benefits of using computerized adaptive testing (CAT) to improve the efficiency and effectiveness of measuring amphetamine craving in clinical subjects. By reducing the number of items needed while maintaining reliability, this approach could ultimately lead to more personalized and targeted treatments for individuals with drug addiction.

show abstract

Developing Computerized Adaptive Testing for a National Health Professionals Exam: An Attempt from Psychometric Simulations

Xu,

Jiang,

Han

et al. 2023

Perspect Med Educ

View full text Add to dashboard Cite

Introduction: The accurate assessment of health professionals’ competence is critical for ensuring public health safety and quality of care. Computerized Adaptive Testing (CAT) based on the Item Response Theory (IRT) has the potential to improve measurement accuracy and reduce respondent burden. In this study, we conducted psychometric simulations to develop a CAT for evaluating the candidates’ competence of health professionals. Methods: The initial CAT item bank was sourced from the Standardized Competence Test for Clinical Medicine Undergraduates (SCTCMU), a nationwide summative test in China, consisting of 300 multiple-choice items. We randomly selected response data from 2000 Chinese clinical medicine undergraduates for analysis. Two types of analyses were performed: first, evaluating the psychometric properties of all items to meet the requirements of CAT; and second, conducting multiple CAT simulations using both simulated and real response data. Results: The final CAT item bank consisted of 121 items, for which item parameters were calculated using a two-parameter logistic model (2PLM). The CAT simulations, based on both simulated and real data, revealed sufficient marginal reliability (coefficient of marginal reliability above 0.750) and criterion-related validity (Pearson’s correlations between CAT scores and aggregate scores of the SCTCMU exceeding 0.850). Discussion: In national-level medical education assessment, there is an increasing need for concise yet valid evaluations of candidates’ competence of health professionals. The CAT developed in this study demonstrated satisfactory reliability and validity, offering a more efficient assessment of candidates’ competence of health professionals. The psychometric properties of the CAT could lead to shorter test durations, reduced information loss, and a decreased testing burden for participants.

show abstract

Development and Validation of an Item Bank for Drug Dependence Measurement Using Computer Adaptive Testing

Cited by 3 publications

References 55 publications

Developing an Item Bank of Computerized Adaptive Testing for Eating Disorders in Chinese University Students

Developing an Item Bank of Computerized Adaptive Testing for Eating Disorders in Chinese University Students

Applying computerized adaptive testing to the Desires for Speed Questionnaire in the Chinese population: A simulation study.

Developing Computerized Adaptive Testing for a National Health Professionals Exam: An Attempt from Psychometric Simulations

Contact Info

Product

Resources

About