Integrating domain knowledge with user eye gaze in automated word acquisition for conversational interfaces

Abstract

Most conversation systems tend to fail when unexpected words are encountered. To overcome this problem, conversational systems must be able to learn new words automatically during human machine conversation. Motivated by psycholinguistic findings on eye gaze and human language processing, we have developed several techniques to incorporate human eye gaze for automatic word acquisition in multimodal conversational systems. This paper presents our new results on integrating eye gaze and domain knowledge in word acquisition in a 3D virtual world application. Our results demonstrate that the incorporation of domain knowledge can potentially improve word acquisition performance.