False positives associated with responder/non-responder analyses based on motor evoked potentials

Abstract

Background: A trend in the non-invasive brain stimulation literature is to assess the outcome of an intervention using a responder analysis whereby participants are di-or trichotomised in order that they may be classified as either responders or non-responders. Objective: Examine the extent of the Type I error in motor evoked potential (MEP) data subjected to responder analyses. Methods: Seven sets of 30 MEPs were recorded from the first dorsal interosseous muscle in 52 healthy volunteers. Four classification techniques were used classify the participants as responders or nonresponders: (1) the two-step cluster analysis, (2) Dichotomised thresholding, (3) relative method and (4) baseline variance method. Results: Despite the lack of any intervention, a significant number of participants were classified as responders (21-71%). Conclusion: This study highlights the very large Type I error associated with dichotomising continuous variables such as the TMS MEP.

Interindividual Variability of Lower-Limb Motor Cortical Plasticity Induced by Theta Burst Stimulation

“…This systematic review suggests that cluster analysis is an effective approach for classifying responders and non-responders within the analysis (Pellegrini et al, 2018). In addition, cluster analysis has been reported to produce the fewest number of false positive classifications for several different classification methods (van de Ruit and Grey, 2019). Therefore, our results will likely be useful for defining responders in future works.…”

“…The distance and cluster criteria required for the analysis were defined as the Euclidian distance and Bayesian information criterion, respectively. The optimal number of clusters was determined automatically based on the Bayesian information criterion with no limit (Schwartz, 1978;van de Ruit and Grey, 2019). Based on a previous report (López-Alonso et al, 2014), each cluster was classified as "responders" or "non-responders."…”

Letter to the editor: No influence of static magnetic stimulation applied for 30 minutes over the human M1 on corticospinal excitability

“…These results emphasize the crucial need to implement strategies to minimize variability [6] to firmly establish the reliability and efficiency of tSMS. Such strategies include performing sample size calculations [7], using neuronavigation to ensure stable coil positioning [6], and refraining from pre-selecting or pooling participants on the basis of being responders or not to a given NIBS intervention [8]. Finally, as the relevance of MEP amplitude alterations in drawing inferences on cortical functioning remains poorly understood [9], the failure of tSMS to alter CSE does not necessarily entail that tSMS has no influence on cortical functioning [10].…”

Ten minutes of transcranial static magnetic field stimulation does not reliably modulate motor cortex excitability

“…Moreover, in the subject´s first session, independent of the oddball task, MEPs tended to increase over time post stimulation, whereas there was no change in the subject´s second sessions. The tendency of MEP amplitudes to increase over time in the absence of any intervention was observed before [35][36][37]. However, in our data this effect is mainly driven by a subsample of four subjects (see Fig 3).…”

“…However, less than half of the subjects responded as expected, and it was suggested that inter-individual variability has to be thoroughly addressed in the field of brain stimulation. Moreover, recently it was shown that cluster analyses based on MEPs might be sensitive for false positive results [37]. In that study, following four different classification techniques a significant number of subjects were classified as responders despite any intervention.…”