Maximizing the Potential of Patient-Reported Assessments by Using the Open-Source Concerto Platform With Computerized Adaptive Testing and Machine Learning

Harrison, Conrad; Loe, Bao Sheng; Lis, Przemysław; Sidey‐Gibbons, Chris

doi:10.2196/20950

Cited by 25 publications

(14 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…During the CAT simulation, we note items 1 (money in your savings), 2 (money you owe), 4 (stress level about finances), and 5 (ability to pay bills) were the most frequently used. These items may be indicative of depleted coping resources and entering into a phase of increased financial toxicity, which is consistent with prior studies [ 24 , 31 ]. The strong correlation between factor score of items 1,2,4,5 and that of fixed 12-item scale makes the use of ENRICh-4 version possible.…”

Section: Discussionsupporting

confidence: 90%

Short-form adaptive measure of financial toxicity from the Economic Strain and Resilience in Cancer (ENRICh) study: Derivation using modern psychometric techniques

et al. 2022

Self Cite

View full text Add to dashboard Cite

Objectives This study sought to evaluate advanced psychometric properties of the 15-item Economic Strain and Resilience in Cancer (ENRICh) measure of financial toxicity for cancer patients. Methods We surveyed 515 cancer patients in the greater Houston metropolitan area using ENRICh from March 2019 to March 2020. We conducted a series of factor analyses alongside parametric and non-parametric item response theory (IRT) assessments using Mokken analysis and the graded response model (GRM). We utilized parameters derived from the GRM to run a simulated computerized adaptive test (CAT) assessment. Results Among participants, mean age was 58.49 years and 278 (54%) were female. The initial round factor analysis results suggested a one-factor scale structure. Negligible levels of differential item functioning (DIF) were evident between eight items. Three items were removed due to local interdependence (Q3>+0.4). The original 11-point numerical rating scale did not function well, and a new 3-point scoring system was implemented. The final 12-item ENRICh had acceptable fit to the GRM (p<0.001; TLI = 0.94; CFI = 0.95; RMSEA = 0.09; RMSR = 0.06) as well as good scalability and dimensionality. We observed high correlation between CAT version scores and the 12-item measure (r = 0.98). During CAT, items 2 (money you owe) and 4 (stress level about finances) were most frequently administered, followed by items 1 (money in savings) and 5 (ability to pay bills). Scores from these four items alone were strongly correlated with that of the 12-item ENRICh (r = 0.96). Conclusion These CAT and 4-item versions provide options for quick screening in clinical practice and low-burden assessment in research.

show abstract

Section: Discussionsupporting

confidence: 90%

Short-form adaptive measure of financial toxicity from the Economic Strain and Resilience in Cancer (ENRICh) study: Derivation using modern psychometric techniques

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…Analysis of the group data found that VA was associated with greater activity limitation and worse mobility, while poorer VF mean deviation was associated with worse emotional well-being. These pilot results provide real-world evidence of the implementation feasibility of GlauCAT™ in routine clinical care enabling real-time discussion of results [ 49 ] and subsequent analysis of group data to better understand the factors associated with HRQoL outcomes in glaucoma patients [ 48 ].…”

Section: Discussionmentioning

confidence: 99%

Measuring glaucoma quality of life in an Asian population using item banking: psychometric evaluation and computerized adaptive testing simulations

et al. 2023

Self Cite

View full text Add to dashboard Cite

Purpose To assess the psychometric properties of glaucoma-specific health-related quality of life (HRQoL) item banks (IBs), and explore their efficiency using computerized adaptive testing (CAT) simulations. Methods In this cross-sectional, clinical study, 300 Asian glaucoma patients answered 221 items within seven IBs: Ocular Comfort Symptoms (OS); Activity Limitation (AL); Lighting (LT); Mobility (MB); Glaucoma Management (GM); Psychosocial (PSY); and Work (WK). Rasch analysis was conducted to assess each IB’s psychometric properties (e.g., item “fit” to the construct; unidimensionality) and a set of analytic performance criteria guiding decision making relating to retaining or dropping domains and items was employed. CAT simulations determined the mean number of items for ‘high’ and ‘moderate’ measurement precision (stopping rule: SEM 0.3 and 0.387, respectively). Results Participants’ mean age was 67.2 ± 9.2 years (62% male; 87% Chinese). LT, MB, and GM displayed good psychometric properties overall. To optimize AL’s psychometric properties, 16 items were deleted due to poor “fit”, high missing data, item bias, low discrimination and/or a low clinical/patient importance rating. To resolve multidimensionality in PSY, we rehomed 16 items into a “Concern (CN)” domain. PSY and CN required further amendment, including collapsing of response categories, and removal of poorly functioning items ( N = 7). Due to poor measurement precision, low applicability and high ceiling effect, low test information indices, and low item separation index the WK IB was not considered further. In CAT simulations on the final seven IBs ( n = 182 items total), an average of 12.1 and 15.7 items per IB were required for moderate and high precision measurement, respectively. Conclusions After reengineering our seven IBs, they displayed robust psychometric properties and good efficiency in CAT simulations. Once finalized, GlauCAT™-Asian may enable comprehensive assessment of the HRQoL impact of glaucoma and associated treatments. Supplementary Information The online version contains supplementary material available at 10.1007/s11136-023-03428-8.

show abstract

“…In view of the current push toward value-based clinical care, 9 we have developed an online CAT testing platform (PROMinsight) using Concerto open-source software that can be implemented on any smart device, enabling real-time scoring and reporting of data. 41 Indeed, a pilot study conducted in glaucoma clinics at Massachusetts Eye and Ear Institute, where 216 glaucoma patients completed six GlauCAT tests on a tablet device while waiting to see their treating doctor, 42 , 43 showed that patients took just 8 minutes and 5 seconds (median) to complete all six GlauCAT tests, 43 providing real-world evidence of the feasibility of implementing GlauCAT in routine clinical care.…”

Section: Discussionmentioning

confidence: 99%

Psychometric Evaluation of Glaucoma Quality of Life Item Banks (GlauCAT) and Initial Assessment Using Computerized Adaptive Testing

Fenwick

Khadka

et al. 2022

Trans. Vis. Sci. Tech.

View full text Add to dashboard Cite

Purpose To evaluate the psychometric properties of glaucoma-specific quality of life (QoL) item banks (GlauCAT) and assess their performance using computerized adaptive testing (CAT) simulations. Methods In this cross-sectional study, 293 participants with glaucoma (mean age ± SD, 70.7 ± 13.2 years; 45% female) answered 342 items in 12 QoL item banks (IBs): Activity Limitation (AL); Driving (DV); Convenience (CV); Economic (EC); Emotional (EM); General Symptoms (GS); Health Concerns (HC); Lighting (LT); Mobility (MB); Ocular Surface Symptoms (OS); Social (SC); and Visual Symptoms (VS). These IBs were assessed using Rasch analysis, and CAT simulations with 1000 simulated respondents were utilized to determine the average number of items to be administered to achieve moderate and high precision levels. Results The AL, DV, EM, HC, LT, MB, EC, OS, SC, and VS IBs required relatively minor amendments to achieve satisfactory psychometric fit. To resolve multidimensionality, we split CV into Treatment Convenience (TCV) and General Convenience (GCV). Due to poor measurement precision, the GS IB was not pursued further. This resulted in 12 total IBs. In CAT simulations, an average of 3.7 and 7.3 items per IB were required to attain measurement at moderate and high precision, respectively. Conclusions Following rigorous psychometric assessment, we developed 12 valid glaucoma-specific QoL domains that can obtain highly precise person measure estimates using a small number of items. Translational Relevance GlauCAT will enable researchers and clinicians to quickly and comprehensively assess the impact of glaucoma and its associated interventions across a range of QoL domains.

show abstract

Maximizing the Potential of Patient-Reported Assessments by Using the Open-Source Concerto Platform With Computerized Adaptive Testing and Machine Learning

Cited by 25 publications

References 26 publications

Short-form adaptive measure of financial toxicity from the Economic Strain and Resilience in Cancer (ENRICh) study: Derivation using modern psychometric techniques

Short-form adaptive measure of financial toxicity from the Economic Strain and Resilience in Cancer (ENRICh) study: Derivation using modern psychometric techniques

Measuring glaucoma quality of life in an Asian population using item banking: psychometric evaluation and computerized adaptive testing simulations

Psychometric Evaluation of Glaucoma Quality of Life Item Banks (GlauCAT) and Initial Assessment Using Computerized Adaptive Testing

Contact Info

Product

Resources

About