Reclassifying guesses to increase signal-to-noise ratio in psychological experiments

Abstract

Researchers studying the mind often rely on behavioral tasks and differences, either in stimuli or in brain activity, between correct and incorrect trials. However, subjects often guess when they don't know the answer, leading to correct responses that result from the same causes as the incorrect responses: this is a source of noise that remains no matter the number of trials performed by the subjects. This paper presents a response reclassification procedure to reduce the noise caused by “false” correct responses using an independent source of reclassification evidence. We illustrate the procedure on data from Faghel-Soubeyrand et al. (2019) with response times as reclassification evidence. The reclassification procedure increased signal-to-noise ratio by about 13.5% with little bias. Matlab and Python implementations of the reclassification procedure are freely available.