Spatio-temporal dynamics of automatic processing of phonological information in visual words

Abstract

Sensory-specific cortices appear to be sensitive to information from another modality. Here we investigate whether the human brain automatically extracts the phonological information in visual words in early visual processing. We continuously presented native Chinese speakers peripherally with Chinese homophone characters in an oddball paradigm, while they performed a visual detection task presented in the centre of the visual field. We found the lexical tone phonology embedded in the characters is processed automatically by the brain of native speakers, as revealed by whole-head electrical recordings of the mismatch negativity (MMN). Source solution further revealed the MMN involved the neural activations from the visual cortex to the auditory cortex (130-460 ms). The spatial-temporal dynamics indicate a visual-auditory interaction in the early, automatic processing of phonological information in visual words.I t is interesting and counterintuitive that sensory-specific cortices appear to be sensitive to information from another modality 1,2 . Audiovisual speech processing which involves multisensory integration sites including low level auditory and visual sensory regions 3,4 is ecologically important in daily communications. Empirical evidence from behavioural and neuroimaging studies demonstrated that in audiovisual speech processing, human auditory cortex responds to visual information and vice versa. A typical example of audiovisual speech processing is the integration of a written and a spoken language. Learning the correspondence between a spoken word and its written form is a prerequisite for the development of reading and writing skills 5 . Behavioural studies of written word recognition demonstrated the role of auditory phonology in visual word processing [6][7][8][9][10] . Some recent neuroimaging studies showed that phonological processing during visual word recognition is governed mainly by the left brain hemisphere including left lateralized supramarginal gyrus and left lateralized inferior frontal cortex, as well as bilateral superior temporal gyri [11][12][13][14] . In event-related potential (ERP) studies of phonological processing in written words, the majority of research focused on N400 or even later stage components [15][16][17][18] . A few other ERP studies pointed to relatively early phonological effects on the early components such as P2 and N2 [19][20][21][22] .The issue of whether or not phonological information embedded in written words can be automatically extracted by the brain and rapidly stored in visual sensory memory has remained virtually unexplored. To address this issue, an effective approach to isolate the brain response component contributed by the cognitive processing at an early, preattentive stage with a high temporal resolution is essential. To this end, the mismatch negativity (MMN) can be a potential tool. MMN has been suggested an index of early, automatic encoding of the changes of regularities in both of the auditory 23-25 and visual modality 26 . In audition, MMN ...

Rapid implicit extraction of abstract orthographic patterns of Chinese characters during reading

“…We did not observe evidence supporting the detection of violations of implicit abstract patterns of lexical tone in the visual system at low levels. This seems inconsistent with previous studies documenting a rapid, automatic detection of violations of abstract patterns of Chinese character phonology [31,32]. The inconsistency may be attributed to the different difficulties of tone identification tasks.…”

“…The auditory cortex has been documented to have the primitive intelligence to extract the abstract patterns of Chinese lexical tones with various consonants and vowels [42]. In addition, the primitive intelligence to extract phonological features of Chinese characters is not limited to the auditory modality but also extends to the visual modality [31,32]. In this study, we add to the current literature that the visual cortex indeed has the primitive intelligence to extract abstract orthographic patterns, or to be more specific, abstract structural patterns.…”

“…Another reasonable explanation is that the extraction of abstract phonological patterns in the visual cortex could reach only syllabic level but not deeper phonemic level. In other words, abstract syllabic patterns of Chinese characters can be rapidly and automatically extracted as documented by the studies by Wang et al [31,32], but abstract phonemic patterns of Chinese characters may not be extracted in the visual cortex. According to the interactive model of visual word recognition, higher-order linguistic representations such as phonological representations can modulate early orthographic processing, which further modulates earlier visual processing [45].…”

“…Moreover, Wang et al used the vMMN to study the rapid feature extracting procedure of Chinese characters during reading; they discovered significant vMMN effects for infrequent violations of the tonal pattern embedded in standard characters, indicating that abstract phonological patterns of Chinese characters are rapidly extracted in the visual cortex [31]. This vMMN effect still exists even when the stimuli are displayed at the corner of the screen away from the center to which the attention of subjects is attracted by an irrelevant visual task [32]. However, it still lacks evidence to clarify whether the visual cortex can implicitly extract abstract spatial patterns of radical configuration of Chinese characters.…”

“…This paradigm was also used to replicate the findings that abstract phonological patterns of Chinese characters are implicitly extracted in the visual cortex [31,32]. In the session where subjects were instructed to identify the target characters with a deviant tone, we expected to observe differences within in the time range of the vMMN between ERPs in response to standard stimuli and target ones, which represented the active extraction of the abstract tonal pattern.…”