Improved Optical Flow for Gesture-based Human-robot Interaction

Abstract

Gesture interaction is a natural way of communicating with a robot as an alternative to speech. Gesture recognition methods leverage optical flow in order to understand human motion. However, while accurate optical flow estimation (i.e., traditional) methods are costly in terms of runtime, fast estimation (i.e., deep learning) methods' accuracy can be improved. In this paper, we present a pipeline for gesture-based human-robot interaction that uses a novel optical flow estimation method in order to achieve an improved speed-accuracy trade-off. Our optical flow estimation method introduces four improvements to previous deep learning-based methods: strong feature extractors, attention to contours, midway features, and a combination of these three. This results in a better understanding of motion, and a finer representation of silhouettes. In order to evaluate our pipeline, we generated our own dataset, MIBURI, which contains gestures to command a house service robot. In our experiments, we show how our method improves not only optical flow estimation, but also gesture recognition, offering a speed-accuracy trade-off more realistic for practical robot applications.

Accurate Realtime Motion Estimation Using Optical Flow on an Embedded System

“…The largest AE of each index shown given by [26] was an expected result of the measurement of the dense optical flow with low robustness. Therefore, we can say that the estimated optical flow used in [27,28] was almost close and presented good accuracy.…”

“…Table 1 shows the average AE retained by applying the optical flow estimation methods on the Yosemite sequence with clouds and without clouds. The results obtained by [27,28] were better than those produced by [26] but the obtained optical flow seemed sparse. Therefore, they were missing many useful details especially local moving information.…”

“…Therefore, they were missing many useful details especially local moving information. According to [28], the method of [27] in visual comparison shows that it could generate a higher robustness than [26]. The largest AE of each index shown given by [26] was an expected result of the measurement of the dense optical flow with low robustness.…”

“…In this table, the comparison of the average AE of our method with three others of [26][27][28] is given. The performance of optical flow estimation of these algorithms was given using the Middlebury dataset image sequences.…”

A Database for Learning Numbers by Visual Finger Recognition in Developmental Neuro-Robotics

“…Gestures represent a form of simulated action that arise from an embodied cognitive system (Hostetter and Alibali, 2008 ; Tsiami et al, 2018 ; Chang et al, 2019 ). In particular, the use of hands and gestures is a very attractive method for providing natural human-computer interaction (Erol et al, 2007 ).…”