Brain correlates of phonological recoding of visual symbols

Abstract

Learning to read involves setting up associations between meaningless visual inputs (V) and their phonological representations (P). Here, we recorded the brain signals (ERPs and fMRI) associated with phonological recoding (i.e., V-P conversion processes) in an artificial learning situation in which participants had to learn the associations between 24 unknown visual symbols (Japanese Katakana characters) and 24 arbitrary monosyllabic names. During the learning phase on Day 1, the strength of V-P associations was manipulated by varying the proportion of correct and erroneous associations displayed during a two-alternative forced choice task. Recording event related potentials (ERPs) during the learning phase allowed us to track changes in the processing of these visual symbols as a function of the strength of V-P associations. We found that, at the end of the learning phase, ERPs were linearly affected by the strength of V-P associations in a time-window starting around 200ms post-stimulus onset on right occipital sites and ending around 345ms on left occipital sites. On Day 2, participants had to perform a matching task during an fMRI session and the strength of these V-P associations was again used as a probe for identifying brain regions related to phonological recoding. Crucially, we found that the left fusiform gyrus was gradually affected by the strength of V-P associations suggesting that this region is involved in the brain network supporting phonological recoding processes.

The impact of spelling regularity on handwriting production: A coupled fMRI and kinematics study

“…In addition, our results extend the scope of the functional properties of the left IFG and FuG in spelling, (Bitan et al, 2005;Planton et al, 2013;Rapp & Lipka, 2011;Rapp et al, 2016). In reading, the left FuG is thought to represent orthographic information on a visual form (Vinckier et al, 2007), but this view remains controversial (Madec et al, 2016;Price & Delvin, 2003;Rapp & Lipka, 2011). In fact, several studies showed that the left FuG is involved in accessing or storing orthographic information during both reading and spelling (Purcell et al, 2011;Rapp & Lipka, 2011;Tsapkini & Rapp, 2010).…”

Audiovisual Processing of Chinese Characters Elicits Suppression and Congruency Effects in MEG

“…Other research has also indicated that the right parietal region plays a role in the perception of stimuli without any long-term representations arising simultaneously from the multiple sensory modalities (Kamke et al, 2012). The suppression effect in the right parietal and occipital regions at early time window indicates that the unfamiliar audiovisual information processing might rely more on the processing of the visual features (Calvert et al, 2001; Madec et al, 2016). It could be explained by the fact that Finnish participants who are naive to Chinese focused much more attention on the analysis of the spatial information of various strokes comprising the logographic character in order to be able to integrate the audiovisual stimuli.…”

Shared premotor activity in spoken and written communication

“…Contrary to Raij et al (2000) and van Attelveldt et al (2004), we found no reliable heteromodal activation of the superior temporal gyrus/superior temporal sulcus when unimodal visual letters and speech sounds were presented. In addition, although in some cases (most likely when the task demands access to the letter name), the level of activation of the fusiform gyrus is modulated by inputs from upstream regions coding for phonological processing (Dietz et al, 2005;Xue et al, 2006;Madec et al, 2016), our results confirm that speech itself is not sufficient to lead to significant activation of the fusiform gyrus at the level of the visual word form area.…”

“…From that view, the fact that premotor activity was found to be equal in the speaking and writing tasks, and to also be equal but lower in the listening and reading tasks (see Figure 4), might suggest similar articulatory planning processing in the two first motor tasks, together with reduced motor reactivation in the two perceptual tasks. Note that, if this interpretation is correct, the degree of association between the letters and sounds specific to a given language might affect the results (see Madec et al, 2016). French is a non-transparent language where a letter can correspond to several sounds.…”

“…Letters name is considered a component of the neural representation of letters (Rothlein and Rapp, 2014). Behaviorally, phonological information can efficiently prime letter recognition (Arguin and Bub, 1995;Bowers, Vigliocco and Haan, 1998;Ziegler et al, 2000) and modulate the brain's response to visually presented letters (Dietz et al, 2005;Xue et al, 2006;Madec et al, 2016). Reciprocally, when contrasting literate with illiterate adults, literacy was found to modulate activity in the planum temporale in response to spoken inputs (Dehaene et al, 2010), and more generally, in the networks involved in speech processing (Castro-Caldas et al, 1998).…”