Vision-Based Attentiveness Determination Using Scalable HMM Based on Relevance Theory

Abstract

Attention capability is an essential component of human–robot interaction. Several robot attention models have been proposed which aim to enable a robot to identify the attentiveness of the humans with which it communicates and gives them its attention accordingly. However, previous proposed models are often susceptible to noisy observations and result in the robot’s frequent and undesired shifts in attention. Furthermore, most approaches have difficulty adapting to change in the number of participants. To address these limitations, a novel attentiveness determination algorithm is proposed for determining the most attentive person, as well as prioritizing people based on attentiveness. The proposed algorithm, which is based on relevance theory, is named the Scalable Hidden Markov Model (Scalable HMM). The Scalable HMM allows effective computation and contributes an adaptation approach for human attentiveness; unlike conventional HMMs, Scalable HMM has a scalable number of states and observations and online adaptability for state transition probabilities, in terms of changes in the current number of states, i.e., the number of participants in a robot’s view. The proposed approach was successfully tested on image sequences (7567 frames) of individuals exhibiting a variety of actions (speaking, walking, turning head, and entering or leaving a robot’s view). From these experimental results, Scalable HMM showed a detection rate of 76% in determining the most attentive person and over 75% in prioritizing people’s attention with variation in the number of participants. Compared to recent attention approaches, Scalable HMM’s performance in people attention prioritization presents an approximately 20% improvement.

An Open Source Multi-Modal Data-Acquisition Platform for Experimental Investigation of Blended Control of Scale Vehicles

“…As contamination of eye movement and blink artifacts in EEG recording makes the analysis of EEG data more difficult and could result in erroneous findings [9], using video and computer vision algorithms in synchronous with the EEG data can potentially be used to identify and remove ocular and ambulatory artifacts from the EEG data. Computer vision algorithms can also be employed to monitor attention levels directly [10]. Video of Fig.…”

Detection of Factors Affecting State Transition Based on Non-Homogeneous Markov Chain Model

“…where, Z i,t is the state of the ith observation object at the t-th time, a us is the transition probability from state u at occasion t −1 to state S at occasion t In this transition mode, the current state is only related to the state of the previous moment, that is, it satisfies the homogeneous hypothesis. We call this state transition mode homogeneous Markov state transition [10]. However, in many state sequences, the state of the current observation time is not only related to the state of the previous time, but also related to some characteristics of the current time.…”

Hidden Markov Linear Regression Model and its Parameter Estimation

“…Although the state of the hidden Markov model cannot be directly observed, it can be observed through the sequence of observation vectors, and each observation vector is represented by various probability density distributions in various states, and each observation vector is generated by a state sequence of the corresponding probability density distributions. Therefore, the Hidden Markov Model is a double random process, it has a certain number of hidden Markov chains and a set of display random functions, its research goal is to infer unobservable state transition information and distribution information in each state based on the information of observed variables [3], which in recent years has been widely used in wearable device data identification [4], SDN network early data stream matching [5], speech recognition [6], malfunction diagnosis [7], gene recognition [8], etc., and thereby providing us a series of research results.…”