Amyotrophic Lateral Sclerosis–Bulbar Dysfunction Index–Remote: Test–Retest and Interrater Reliability of Candidate Items

Abstract

Purpose: The primary aim of this study was to establish the reliability of candidate items as a step in the development of the Amyotrophic Lateral Sclerosis–Bulbar Dysfunction Index–Remote (ALS-BDI-Remote), a novel tool being developed for the detection and monitoring of bulbar signs and symptoms in remote settings. Method: The set of candidate items included 40 items covering three domains: cranial nerve examination, auditory-perceptual evaluation, and functional assessment. Forty-eight participants diagnosed with ALS and exhibiting a range of bulbar disease severity were included. Data collection for each participant took place on Zoom over three sessions. During Session 1, the participants were instructed to adjust their Zoom settings and to optimize their recording environment (e.g., lighting, background noise). Their cognition and eating were screened to determine their ability to follow instructions and their eligibility to perform the swallowing and chewing tasks. During Session 2, two speech-language pathologists (SLPs) administered the tool consecutively to determine the items' interrater reliability. During Session 3, one of the SLPs readministered the tool within 2 weeks of Session 1 to assess test–retest reliability. The reliability of each item was estimated using weighted kappa and the percentage of agreement. To be considered reliable, the items had to reach a threshold of 0.5 weighted kappa or 80% percentage agreement (if skewed distribution of the scores) for both interrater and test–retest reliability. Results: In total, 33 of the 40 candidate items reached the reliability cutoff for both reliability analyses. All assessment domains included reliable items. Items requiring very good visualization of structures or movements were generally less reliable. Conclusions: This study resulted in the selection of reliable items to be included in the next version of the ALS-BDI-Remote, which will undergo psychometric evaluation (reliability, validity, and responsiveness analyses). Additionally, the results contributed to our understanding of the remote administration of SLP assessments for telehealth applications.