Robot Aided Intelligent Kitchen Assistance System Using Eye Tracking Based on Object Recognition

Abstract

How to get the potential interaction intent is the key to design a personalized intelligent kitchen assistance system based on Human Robot Interaction (HRI), which can facilitate the elder people in their daily life. Eye movement includes abundant user intent information, eye tracking technology can be used to obtain and analyze this information, thus identifying user’s potential intent. This paper proposed an eye tracking based object recognition method, which could recognize objects that the user was watching. Firstly, multiple fixation points were clustered into a fixation point that best represented the user’s intent by research on the study of the clustering algorithm. Then the object recognition algorithm based on PP-ShiTu platform was used to obtain the specific location of the object in the scene image. Finally, this paper proposed a fixation based object recognition module, and used the module to design and implement a personalized intelligent kitchen assistance system, and verified the effectiveness through experiments. The experimental results showed that when there was only single food recognition, the average recognition accuracy of the personalized intelligent kitchen assistance system reached over 80%, and the average time delay of the system was about 1400 ms.