Testing the Validity and Reliability of a Standardized Virtual Examination for Concussion

Abstract

Background and ObjectivesWe determined inter-modality (in-person vs telemedicine examination) and inter-rater agreement for telemedicine assessments (2 different examiners) using the Telemedicine Buffalo Concussion Physical Examination (Tele-BCPE), a standardized concussion examination designed for remote use. MethodsPatients referred for an initial evaluation for concussion were invited to participate. Participants had a brief initial assessment by the treating neurologist. After a patient granted informed consent to participate in the study, the treating neurologist obtained a concussion-related history before leaving the examination room. Using the Tele-BCPE, 2 virtual examinations in no specific sequence were then performed from nearby rooms by the treating neurologist and another neurologist. After the 2 telemedicine examinations, the treating physician returned to the examination room to perform the in-person examination. Intraclass correlation coefficients (ICC) determined inter-modality validity (in-person vs remote examination by the same examiner) and inter-rater reliability (between remote examinations done by 2 examiners) of overall scores of the Tele-BCPE within the comparison datasets. Cohen's kappa, κ, measured levels of agreement of dichotomous ratings (abnormality present vs absent) on individual components of the Tele-BCPE to determine inter-modality and inter-rater agreement. ResultsFor total scores of the Tele-BCPE, both inter-modality agreement (ICC = 0.95 [95% CI 0.86-0.98, p < 0.001]) and inter-rater agreement (ICC = 0.88 [95% CI 0.71-0.95, p < 0.001]) were reliable (ICC >0.70). There was at least substantial inter-modality agreement (κ ≥ 0.61) for 25 of 29 examination elements. For inter-rater agreement (2 telemedicine examinations), there was at least substantial agreement for 8 of 29 examination elements. DiscussionOur study demonstrates that the Tele-BCPE yielded consistent clinical results, whether conducted in-person or virtually by the same examiner, or when performed virtually by 2 different examiners. The Tele-BCPE is a valid indicator of neurologic examination findings as determined by an in-person concussion assessment. The Tele-BCPE may also be performed with excellent levels of reliability by neurologists with different training and backgrounds in the virtual setting. These findings suggest that a combination of in-person and telemedicine modalities, or involvement of 2 telemedicine examiners for the same patient, can provide consistent concussion assessments across the continuum of care.