Multimodal Person Recognition for Human-Vehicle Interaction

Abstract

Next-generation vehicles will undoubtedly feature biometric person recognition as part of an effort to improve the driving experience. Today's technology prevents such systems from operating satisfactorily under adverse conditions. A proposed framework for achieving person recognition successfully combines different biometric modalities, borne out in two case studies. O ver the past 30 years, the field of biometric person recognition-recognizing individuals according to their physical and behavioral characteristics-has undergone significant progress. Next-generation human−vehicle interfaces will likely incorporate biometric person recognition, using speech, video, images, and analog driver behavior signals to provide more efficient and safer vehicle operation, as well as pervasive and secure in-vehicle communication. Yet, technical and deployment limits hamper these systems' ability to perform satisfactorily in real-world settings under adverse conditions. For instance, environmental noise and changes in acoustic and microphone conditions can significantly degrade speaker recognition performance. Similarly, factors such as illumination and background variation, camera resolution and angle, and facial expressions contribute to performance loss in visually identifying a person. Biometric person recognition in vehicles is especially likely to challenge researchers because of difficulties posed by the vehicle's interior compartment as well as by economics.In this article, we present an overview of multimodal in-vehicle person recognition technologies. We demonstrate, through a discussion of our proposed framework, that the levels of accuracy required for person recognition can be achieved by fusing multiple modalities. We discuss techniques and prominent research efforts, and we present the results of two case studies we conducted. The sidebar, "Solutions for In-Vehicle Person Recognition," discusses related work. Why in-vehicle person recognition?To improve the driving experience, making it better and safer, manufacturers are making vehicles smarter. As vehicles become smarter, information processing from vehicle sensors will become more complex, as will vehicle personalization, which involves adapting the driver and passenger compartment for safer driving and travel.Personalization features will, for example, increase vehicle safety by determining whether the person behind the wheel is an authorized driver (say, the vehicle's legal owner). If so, the driver will be able to operate the vehicle. If the individual isn't authorized, the vehicle will prevent operation and could communicate with authorities to report the incident and initiate an enforcement procedure, such as preventing the driver from starting the ignition.Personalization will promote safe driving by monitoring a driver's behavior. Once the human−vehicle interface has identified a driver and determined road and traffic conditions, the vehicle can monitor the driver's behavioral signals from braking, accelerating, or swerving. With these signals, the hum...

Predicting car states through learned models of vehicle dynamics and user behaviours

“…The first category concentrates on identifying individual drivers or grouping a set of drivers through a suitable set of inputs [8], [9]. Erzin et al [10] presented a system that can detect if the driver is drunk, distracted or fatigued through the user behaviour, in order for assistive action to be taken.…”

Real-time reliability measure-driven multi-hypothesis tracking using 2D and 3D features

“…Therefore, the object features could be weighted according to their reliability to generate a new dynamics model which takes advantage able to cope with noisy, erroneous or missing data. Reliability measures have been used in the literature for focusing on the relevant information [39][40][41], allowing more robust processing. Nevertheless, these measures have been only used for specific tasks of the video understanding process.…”

Sensor-Actuator Supported Implicit Interaction in Driver Assistance Systems

“…53 Difference threshold is commonly known as Just Noticeable Difference (JND). (iii) Range of weight: A system using the weight of a driver, e. g. for person identification, has to be defined in a way to cover all persons possible as drivers (for example a range between 40kg and 180kg).…”