The Effect of Visual Articulatory Information on the Neural Correlates of Non-native Speech Sound Discrimination

Abstract

Behavioral studies have shown that the ability to discriminate between non-native speech sounds improves after seeing how the sounds are articulated. This study examined the influence of visual articulatory information on the neural correlates of non-native speech sound discrimination. English speakers' discrimination of the Hindi dental and retroflex sounds was measured using the mismatch negativity (MMN) eventrelated potential, before and after they completed one of three 8-min training conditions. In an audio-visual speech training condition (n = 14), each sound was presented with its corresponding visual articulation. In one control condition (n = 14), both sounds were presented with the same visual articulation, resulting in one congruent and one incongruent audio-visual pairing. In another control condition (n = 14), both sounds were presented with the same image of a still face. The control conditions aimed to rule out the possibility that the MMN is influenced by non-specific audio-visual pairings, or by general exposure to the dental and retroflex sounds over the course of the study. The results showed that audio-visual speech training reduced the latency of the MMN but did not affect MMN amplitude. No change in MMN amplitude or latency was observed for the two control conditions. The pattern of results suggests that a relatively short audiovisual speech training session (i.e., 8 min) may increase the speed with which the brain processes non-native speech sound contrasts. The absence of a training effect on MMN amplitude suggests a single session of audio-visual speech training does not lead to the formation of more discrete memory traces for non-native speech sounds. Longer and/or multiple sessions might be needed to influence the MMN amplitude.

Mismatch Negativity (MMN): Komponen Event-Related Potentials (ERP) sebagai Penanda Aktivitas Otomatis Otak dalam Mendeteksi Perubahan

“…Secara umum MMN digunakan sebagai indeks untuk akurasi dan kemampuan mendiskriminasi suara. Beberapa contoh penelitian di antaranya: pada individu dengan implan koklea (Engstrom et al, 2021;Fernandes et al, 2019;Näätänen et al, 2017), kemampuan diskriminasi pada suara ucap dari bahasa asing (Plumridge et al, 2020), pengaruh beban mental pada kondisi simulasi penerbangan (Wanyan et al, 2018), diskriminasi suara ucap pada anak-anak dan kaitannya dengan kemampuan kognitif (Kuuluvainen et al, 2016).…”