The Hard Limits of Decoding Mental States: The Decodability of fMRI

Abstract

SUMMARYHigh-profile studies claim to assess mental states across individuals using multi-voxel decoders of brain activity. The fixed, fine-grained, multi-voxel patterns in these “optimized” decoders are purportedly necessary for discriminating between, and accurately identifying, mental states. Here, we present compelling evidence that the efficacy of these decoders is overstated. Across a variety of tasks, decoder patterns were not necessary. Not only were “optimized decoders” spatially imprecise and 90% redundant, but they also performed similarly to simpler decoders, built from average brain activity. We distinguish decoder performance when used for discriminating between, in contrast to identifying, mental states, and show even when discrimination performance is strong, identification can be poor. Using similarity rules, we derived novel and intuitive discriminability metrics that capture 95% and 68% of discrimination performance within- and across-subjects, respectively. These findings demonstrate that current across-subject decoders remain inadequate for real-life decision making.

Decoding pain from brain activity

“…Despite the rapid growth of neuroimaging and electrophysiological data, it remains difficult to draw a consensus or reconcile conflicting findings due to the variability in task design or experimental setup. It is also worth noting the fundamental limits of 'decodability' of brain signals, regardless of fMRI (Jabakhanji et al 2020) or EEG (Parvizi and Kastner 2018). Because of potential overfitting, supervised decoding results based on a small sample size (<100) should be cautioned with their result interpretation (Varoquaux 2018, Poldrack et al 2020.…”