This study investigated the effects of intermodal timing differences and speed differences on word intelligibility of auditory-visual speech. Words were presented under visual-only, auditory-only, and auditory-visual conditions. Two types of auditory-visual conditions were used: asynchronous and expansion conditions. In the asynchronous conditions, the audio lag was 0-400 ms. In the expansion conditions, the auditory signal was time expanded (0-400 ms), whereas the visual signal was kept at the original speed. Results showed that word intelligibility was higher in the auditory-visual conditions than in the auditory-only condition. The results of auditory-visual benefit revealed that the benefit at the end of words declined as the amount of time expansion increased, although it did not decline in the asynchronous conditions.

This study investigated effects of asynchrony between speech signal and moving image of talker's face induced by time-expansion of the speech signal on speech intelligibility. Word intelligibility test was performed to younger listeners. Japanese 4-mora words were uttered by a female speaker. Each word was processed with STRAIGHT software to expand the speech signal by from 0 to 400 ms. These signals were combined with moving image of talker's face which was kept at original speed. This test was performed under three conditions: visual-only, auditory-only, and auditory-visual (AV) condition. Results showed that intelligibility scores under AV condition were statistically higher than those under auditory-only condition even when the speech signal was expanded by 400 ms. These results suggest that moving image of talker's face is effective to enhance speech intelligibility if the lag between the speech signal and moving image of talker's face does not exceed 400 ms.Keywords Time-expanded speech · Moving image of talker's face · Word intelligibillity · Audio-visual interaction · Lip-reading