Music reading experience modulates eye movement pattern in English reading but not in Chinese reading

Liao, Weiyan; Li, Sara Tze Kwan; Hsiao, Janet H.

doi:10.1038/s41598-022-12978-9

Cited by 10 publications

(6 citation statements)

References 66 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…H. F. Chan, Suen, et al, 2020; Cho et al, 2022a, 2022b, 2022c), gaze perception (S. K. W. Chan et al, 2022), sustained attention to response (Lee et al, 2021), website viewing (Eckhardt et al, 2013), video viewing (Zheng, Ye, & Hsiao, 2022), scene perception (Hsiao, Lan, et al, 2021), reading (Liao et al, 2022), and visual search (Hsiao, Chan, et al, 2021). EMHMM is based on the assumption that the current fixation in a visual task is conditioned on the previous fixation, and thus eye movements in the task may be considered a Markovian stochastic process, which can be better understood using hidden Markov models (HMMs).…”

Section: Methodsmentioning

confidence: 99%

Differential audiovisual information processing in emotion recognition: An eye-tracking study.

Zheng¹,

Hsiao²

2023

Emotion

View full text Add to dashboard Cite

Recent research has suggested that dynamic emotion recognition involves strong audiovisual association; that is, facial or vocal information alone automatically induces perceptual processes in the other modality. We hypothesized that different emotions may differ in the automaticity of audiovisual association, resulting in differential audiovisual information processing. Participants judged the emotion of a talking-head video under audiovisual, video-only (with no sound), and audio-only (with a static neutral face) conditions. Among the six basic emotions, disgust had the largest audiovisual advantage over the unimodal conditions in recognition accuracy. In addition, in the recognition of all the emotions except for disgust, participants' eye-movement patterns did not change significantly across the three conditions, suggesting mandatory audiovisual information processing. In contrast, in disgust recognition, participants' eye movements in the audiovisual condition were less eyes-focused than the video-only condition and more eyes-focused than the audio-only condition, suggesting that audio information in the audiovisual condition interfered with eye-movement planning for important features (eyes) for disgust. In addition, those whose eye-movement pattern was affected less by concurrent disgusted voice information benefited more in recognition accuracy. Disgust recognition is learned later in life and thus may involve a reduced amount of audiovisual associative learning. Consequently, audiovisual association in disgust recognition is less automatic and demands more attentional resources than other emotions. Thus, audiovisual information processing in emotion recognition depends on the automaticity of audiovisual association of the emotion resulting from associative learning. This finding has important implications for real-life emotion recognition and multimodal learning.

show abstract

Section: Methodsmentioning

confidence: 99%

Differential audiovisual information processing in emotion recognition: An eye-tracking study.

Zheng¹,

Hsiao²

2023

Emotion

View full text Add to dashboard Cite

show abstract

“…Eye movements were recorded using an SR Research EyeLink 1000 Plus eye tracker (sampling rate = 1000 Hz). EyeLink default settings for cognitive research were used in the data acquisition (saccade velocity threshold: 30°/sec; saccade acceleration threshold: 8000°/sec 2 ; saccade motion threshold: 0.1°) 27 , 28 . All stimuli were presented at a distance of 65 cm from the participants on a 21-inch CRT monitor (SONY MuLtiscanG520) with a resolution of 1024 × 768 pixels and refresh rate of 140 Hz.…”

Section: Methodsmentioning

confidence: 99%

Effects of syllable boundaries in Tibetan reading

Wang

Zeng

Han

et al. 2023

Sci Rep

View full text Add to dashboard Cite

Interword spaces exist in the texts of many languages that use alphabetic writing systems. In most cases, interword spaces, as a kind of word boundary information, play an important role in the reading process of readers. Tibetan also uses alphabetic writing, its text has no spaces between words as word boundary markers. Instead, there are intersyllable tshegs (“ ”), which are superscript dots. Interword spaces play an important role in reading as word boundary information. Therefore, it is interesting to investigate the role of tshegs and what effect replacing tshegs with spaces will have on Tibetan reading. To answer these questions, Experiment 1 was conducted in which 72 Tibetan undergraduates read three-syllable-boundary conditions (normal, spaced, and untsheged). However, in Experiment 1, because we performed the experimental operations of deleting tshegs and replacing tshegs, the spatial information distribution of Tibetan sentences under different operating conditions was different, which may have a certain potential impact on the experimental results. To rule out the underlying confounding factor, in Experiment 2, 58 undergraduates read sentences for both untsheged and alternating-color conditions. Overall, the global and local analyses revealed that tshegs, spaces, and alternating-color markers as syllable boundaries can help readers segment syllables in Tibetan reading. In Tibetan reading, both spaces and tshegs are effective visual syllable segmentation cues, and spaces are more effective visual syllable segmentation cues than tshegs.

show abstract

“…Eye-tracking provides a useful method to record every moment of eye movement during reading, based on which the cognitive process of the reader can be inferred 14 . Successful reading comprehension involves two stages of information processing.…”

Section: Literature Reviewmentioning

confidence: 99%

“…Methodologically, while previous studies on reading and BGM were mostly based on self-report measures and reading tests 3 – 5 , 10 – 12 , the development of eye-tracking technology provides a more objective approach with fine-grained eye movement data 13 , 14 . So far few studies have employed eye-tracking technology to probe how BGM impacts the reading process 10 , 15 , 16 , and even fewer delved into finer eye-movement measures such as those in lexical and post-lexical levels 16 , 17 .…”

Section: Introductionmentioning

confidence: 99%

Studying the effect of self-selected background music on reading task with eye movements

Que

Zheng

Hsiao

et al. 2023

Sci Rep

View full text Add to dashboard Cite

Using background music (BGM) during learning is a common behavior, yet whether BGM can facilitate or hinder learning remains inconclusive and the underlying mechanism is largely an open question. This study aims to elucidate the effect of self-selected BGM on reading task for learners with different characteristics. Particularly, learners’ reading task performance, metacognition, and eye movements were examined, in relation to their personal traits including language proficiency, working memory capacity, music experience and personality. Data were collected from a between-subject experiment with 100 non-native English speakers who were randomly assigned into two groups. Those in the experimental group read English passages with music of their own choice played in the background, while those in the control group performed the same task in silence. Results showed no salient differences on passage comprehension accuracy or metacognition between the two groups. Comparisons on fine-grained eye movement measures reveal that BGM imposed heavier cognitive load on post-lexical processes but not on lexical processes. It was also revealed that students with higher English proficiency level or more frequent BGM usage in daily self-learning/reading experienced less cognitive load when reading with their BGM, whereas students with higher working memory capacity (WMC) invested more mental effort than those with lower WMC in the BGM condition. These findings further scientific understanding of how BGM interacts with cognitive tasks in the foreground, and provide practical guidance for learners and learning environment designers on making the most of BGM for instruction and learning.

show abstract

Music reading experience modulates eye movement pattern in English reading but not in Chinese reading

Cited by 10 publications

References 66 publications

Differential audiovisual information processing in emotion recognition: An eye-tracking study.

Differential audiovisual information processing in emotion recognition: An eye-tracking study.

Effects of syllable boundaries in Tibetan reading

Studying the effect of self-selected background music on reading task with eye movements

Contact Info

Product

Resources

About