Practices and pitfalls in inferring neural representations

Abstract

A B S T R A C TA key challenge for cognitive neuroscience is deciphering the representational schemes of the brain. Stimulusfeature-based encoding models are becoming increasingly popular for inferring the dimensions of neural representational spaces from stimulus-feature spaces. We argue that such inferences are not always valid because successful prediction can occur even if the two representational spaces use different, but correlated, representational schemes. We support this claim with three simulations in which we achieved high prediction accuracy despite systematic differences in the geometries and dimensions of the underlying representations. Detailed analysis of the encoding models' predictions showed systematic deviations from ground-truth, indicating that high prediction accuracy is insufficient for making representational inferences. This fallacy applies to the prediction of actual neural patterns from stimulus-feature spaces and we urge caution in inferring the nature of the neural code from such methods. We discuss ways to overcome these inferential limitations, including model comparison, absolute model performance, visualization techniques and attentional modulation. IntroductionA key challenge for cognitive neuroscience is to understand the neural code that underlies the encoding and representation of sensory, motor, spatial, emotional, semantic and other types of information. To decipher the representational schemes of the brain, researchers often employ neuroimaging techniques such as functional magnetic resonance imaging (fMRI). fMRI measures the blood oxygenation level-dependent (BOLD) activation in the brain that is elicited when participants engage with different stimuli. The neural representation underlying each stimulus is assumed to have measurable but complex effects on the BOLD activation patterns. In order to understand what those patterns of activity can tell us about how the brain processes and represents information, researchers have used various analytical tools such as univariate subtraction methods, multivariate pattern (MVP) classification, representational similarity analysis (RSA) and, recently, explicit stimulus-feature-based encoding and decoding models (for reviews, see Davis and Poldrack, 2013, Haxby et al., 2014, or Naselaris et al., 2011. Despite their differences, all of these methods have the same goal -to quantify how changes in task conditions and the properties of the stimuli relate to changes in BOLD activation and vice versa. One way in which these methods differ is in how they achieve that mapping and in what inferences they allow us to draw.In this article, we review some of the known inferential limitations of existing fMRI analysis methods and we highlight an often-overlooked issue in interpreting results from stimulus-feature-based encoding and decoding models. The latter are steadily becoming the de facto gold standard for investigating neural representational spaces (Haxby et al., 2014;Naselaris and Kay, 2015). Using simulated data with known representati...

Representational Organization of Novel Task Sets during Proactive Encoding

“…Importantly, the RSA results could be influenced by other variables statistically related to our manipulations (Popov et al, 2018), such as instructions' length and speed of responses, which differed slightly between conditions. To examine their influence on the results, we performed an additional multiple regression analysis taking both variables into account.…”

An Integrated Neural Decoder of Linguistic and Experiential Meaning

“…However, demonstrating that text-based models do not contain aspects of experiential information does not also entail that the missing information is relevant for explaining semantic brain activity. Consequently, we conduct our forthcoming analyses by testing for brain activity that is explainable using the experiential model but not using the text-based model and vice versa (see also Anderson et al, 2015;Popov et al, 2018). This approach takes the assumption that the text-based model accurately captures all semantic information that can be extracted from text alone.…”

A Guide To Representational Similarity Analysis for Social Neuroscience

“…high-status vs low-status), the correlations will not explain all of the shared variances between stimuli. One solution to this may be to direct attention during a task to specific attributes that can tell the researcher how information is represented (Nastase et al, 2017;Popov et al, 2018). Another solution would be to use other non-correlation-based representational analyses (e.g.…”