A compositional letter code in high-level visual cortex explains how we read jumbled words

Abstract

1We raed jubmled wrods effortlessly, yet the visual representations underlying 2 this remarkable ability remain unknown. Here, we show that well-known principles of 3 neural object representations can explain orthographic processing. We constructed a 4 population of neurons whose responses to single letters matched perception, and 5 whose responses to multiple letters was a weighted sum of its responses to single 6 letters. This simple compositional letter code predicted human performance both in 7 visual search as well as on explicit word recognition tasks. Unlike existing models of 8 word recognition, this code is neurally plausible, seamlessly integrates letter shape 9 and position, and does not invoke any specialized detectors for letter combinations. 10Our results suggest that looking at a word activates a compositional shape code that 11 enables its efficient recognition. 12 13 SIGNIFICANCE STATEMENT 14Reading is a recent cultural invention, but we are remarkably good at reading 15 words and even jubmeld words. It has so far been unclear whether this ability is due 16 to a representation specialized for letter shapes, or is inherited from basic principles 17 of visual processing. Here we show that a large variety of word recognition phenomena 18 can be explained by well-known principles of object representations, whereby single 19 neurons are selective for the shapes of single letters and respond to longer strings 20 according to a compositional rule. 21Page 7 of 42 RESULTS 87We investigated whether visual word representations can be understood using 88 single letter representations. In Experiment 1, we characterized the shape 89 representation of single letters using visual search and demonstrate how search data 90 can be used to construct a population of neurons whose responses predict perception. 91In Experiment 2, we show how bigram search can be predicted using this neural 92 population together with a simple compositional rule. In Experiment 3, we show that 93 visual search for compound words can be predicted using this neural model. Finally 94 we show that this neural model can account for human performance on jumbled word 95 recognition (Experiment 4) as well as word/nonword discrimination (Experiment 5). 96 97 Experiment 1: Single letter searches 98We recruited 16 subjects to perform an oddball visual search task involving 99 pairs of English uppercase letters, lowercase letters and numbers. Since there were a 100 total of 62 items, subjects performed all possible pairs of searches ( 62 C2 = 1,891 101 searches). An example search is shown in Fig. 2A. Subjects were highly consistent in 102 their responses (split-half correlation between average search times of odd-and even-103 numbered subjects: r = 0.87, p < 0.00005). We calculated the reciprocal of search 104 times for each letter pair which is a measure of distance between them (14). These 105 letter dissimilarities were significantly correlated with subjective dissimilarity ratings 106 reported previously (Section S1).