The bi-modal speech recognition system requires a 2-sample language input for training and for testing algorithms which precisely depicts natural English speech. For the purposes of the audio-visual recordings, a training data base of 264 sentences (1730 words without repetitions; 5685 sounds) has been created. The language sample reflects vowel and consonant frequencies in natural speech. The recording material reflects both the lexical word frequencies and casual speech sound frequencies in the BNC corpus of approx. 100m words. The semantically and syntactically congruent sentences mirror the 100m-word corpus frequencies. The absolute deviation from source sound frequencies is.09% and individual vowel deviation is reduced to a level between 0.0006% (min.) and 0.009% (max.). The absolute consonant deviation is 0.006% and oscillates between 0.00002% (min.) and 0.012% (max.). Similar convergence is achieved in the language sample for testing algorithms (29 sentences; 599 sounds). The post-recording analysis involves the examination of particular articulatory settings which aid visual recognition as well as co-articulatory processes which may affect the acoustic characteristics of individual sounds. Results of bi-modal speech elements recognition employing the language material are included in the paper.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.