Hand Gesture Recognition in Real Time for Automotive Interfaces: A Multimodal Vision-Based Approach and Evaluations

Abstract

In this paper, we develop a vision-based system that employs a combined RGB and depth descriptor to classify hand gestures. The method is studied for a human-machine interface application in the car. Two interconnected modules are employed: one that detects a hand in the region of interaction and performs user classification, and another that performs gesture recognition. The feasibility of the system is demonstrated using a challenging RGBD hand gesture data set collected under settings of common illumination variation and occlusion.Index Terms-Depth cue analysis, driver assistance systems, hand gesture recognition, human-machine interaction, infotainment. I. INTRODUCTIONR ECENT years have seen a tremendous growth in novel devices and techniques for human-computer interaction (HCI). These draw upon human-to-human communication modalities to introduce certain intuitiveness and ease to the HCI. In particular, interfaces incorporating hand gestures have gained popularity in many fields of application. In this paper, we are concerned with the automatic visual interpretation of dynamic hand gestures and study these in a framework of an invehicle interface. A real-time vision-based system is developed, with the goal of robust recognition of hand gestures performed by driver and passenger users. The techniques and analysis presented are applicable to many other application fields requiring hand gesture recognition in visually challenging real-world settings.Motivation for In-Vehicle Gestural Interfaces: In this paper, we are mainly concerned with developing a vision-based hand gesture recognition system that can generalize over different users and operating modes and show robustness under challenging visual settings. In addition to the general study of robust descriptors and fast classification schemes for hand gesture recognition, we are motivated by recent research showing advantages of gestural interfaces over other forms of interaction for certain HCI functionalities.Among tactile, touch, and gestural in-vehicle interfaces, gesture interaction was reported to pose certain advantages over the other two, such as lower visual load, reduced driving errors,

Eye Gaze Controlled Projected Display in Automotive and Military Aviation Environments

“…To facilitate human-machine interaction, researchers have already explored use of haptic feedback [33,34], hand gesture tracking based input [35], personalizing instrument displays and predictive models to help drivers in regular driving tasks like parking [36,37].…”

Python-based Raspberry Pi for Hand Gesture Recognition

“…The main purpose of hand segmentation is to separate the user's hand from the background in the frame [13]. In order to achieve this, different image segmentation algorithms has been used such as thresholding process, which leads to the result shown in " Figure 3".…”

Inverter air conditioner intelligent system design

“…Raheja, Karen Das, Sonia Raheja [4] tarafından yapılan farklı el jestlerinin tanımlanması ile ilgili bir çalışma yapılmıştır. 2014 yılında Eshed Ohn-Bar ve Mohan Manubhai [5] Trivedi tarafından yapılmış olan otomotiv arayüzlerinin kontrolünün gerçek zamanlı el jestleriyle kontrolü ile ilgili bir çalışma gerçekleştirilmiştir. Bir diğer çalışma ise Aralık 2014 tarihinde H. Renuka ve B. Goutam [6] tarafından bilgisayar kontrol panellerinin kontrolü için el jestleri tanıma işlemleri gerçekleştirilmiştir.…”