Investigating the midline effect for visual focus of attention recognition

Sheikhi, Samira; Odobez, Jean-Marc

doi:10.1145/2388676.2388720

Cited by 9 publications

(19 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Experimental results have revealed considerable differences between statistical methods and non-statistical measurements [23][24][25][26]. The former mainly focus on appearance-based measurements, whereas the latter usually consider geometric relationship cues, such as the deviation of the nose from the mid-line and the deviation between the new head pose and the original state.…”

Section: Non-statistical Approachesmentioning

confidence: 99%

Entropy-weighted feature-fusion method for head-pose estimation

Liu

2016

J Image Video Proc.

View full text Add to dashboard Cite

This paper proposes a novel entropy-weighted Gabor-phase congruency (EWGP) feature descriptor for head-pose estimation on the basis of feature fusion. Gabor features are robust and invariant to differences in orientation and illuminance but are not sufficient to express the amplitude character in images. By contrast, phase congruency (PC) functions work well in amplitude expression. Both illuminance and amplitude vary over distinctive regions. Here, we employ entropy information to evaluate orientation and amplitude to execute feature fusion. More specifically, entropy is used to represent the randomness and content of information. For the first time, we seek to utilize entropy as weight information to fuse the Gabor and phase matrices in every region. The proposed EWGP feature matrix was verified on Pointing'04 and FacePix. The experimental results demonstrate that our method is superior to the state of the art in terms of MSE, MAE, and time cost.Keywords: EWGP, Head-pose estimation, Entropy weighted, Gabor, Phase congruency, Feature fusion 1 Review IntroductionVisual focus of attention (VFoA) is emphasized to estimate at what or whom a person is looking and is highly correlated with head-pose estimation [1]. To study headpose estimation, three-dimensional orientation parameters from human head images are explored. Head poses convey an abundance of information in natural interpersonal communication (NIC) and human-computer interaction (HCI) [2]; therefore, an increasing number of researchers is seeking more effective and robust methodologies to estimate head pose. Head poses also play a critical role in artificial intelligence (AI) applications and reveal considerable latent significance of personal intent. For example, people nod their heads to represent understanding in conversations and shake their heads to show dissent, confusion, or consideration. Head orientation with a specific finger-pointing direction generally indicates the place that a person wants to go. The combination of head pose and hand gestures is used to assess the target of an individual's interest [3]. Mutual orientation indicates that people are involved in discussion. If a person shifts the head toward a specific direction, it is highly likely that there is an object of interest in this direction. Therefore, the study of VFoA as an indicator of conversation target in human-computer interaction and facial-expression recognition is increasingly of interest.Analyzing head poses is a natural capability of humans but is difficult for AI. However, head-pose estimation has been researched for years, and the state of the art in headpose estimation can contribute greatly to bridging the gap between humans and AI [4,5]. Head-pose estimation is generally interpreted as the capability to infer orientation relative to the observation camera. For example, head pose is exploited to determine the focus point on the screen based on the gaze direction [6]. The factors influencing the estimation of head pose and their relationships have been introdu...

show abstract

Section: Non-statistical Approachesmentioning

confidence: 99%

Entropy-weighted feature-fusion method for head-pose estimation

Liu

2016

J Image Video Proc.

View full text Add to dashboard Cite

show abstract

“…Often, researchers assume a fixed setting, with people facing the camera at a given distance and set time-independent means manually or through learning [2,4]. In this work, we follow the approach of [9] that leverages on body-head-gaze behavioral studies and better accounts for natural gaze shifts. Accordingly, the means were set dynamically as a combination of the direction in which the person should gaze to look at a target, and of the body orientation Rt (estimated as a proxy through the average of the past head poses).…”

Section: Conversation Aware Vfoa Recognitionmentioning

confidence: 99%

“…Accordingly, the means were set dynamically as a combination of the direction in which the person should gaze to look at a target, and of the body orientation Rt (estimated as a proxy through the average of the past head poses). See [9] for more details. Contextual prior.…”

Section: Conversation Aware Vfoa Recognitionmentioning

confidence: 99%

“…To remove this ambiguity, researchers have explored two directions: (i) the use of other social cues, leveraging on the fact that the recognition of non-verbal cues should not be done in isolation, but jointly, as some behaviors provide context to others. In human-human interaction (mainly meetings), examples include speaker information [10] or higher conversational states [5], that can be complemented with group activity [1]; (ii) the improvement of the prediction of the gaze direction (including eye information) from the head pose, allowing a better association of a head pose with looking at a given target [9].…”

Section: Introductionmentioning

confidence: 99%

“…To improve VFOA recognition, we propose to leverage on two types of robot dialog acts that can affect VFOA expectations: communicative acts (people look more at speakers; and this is particularly true from addressed persons), and lexical (implicit or explicit references to scene object(s)). We propose an Input-Output Hidden Markov Model (IO-HMM) to integrate such information in a recognition process that also leverages on a better head-pose to gaze dynamic mapping process [9]. Experiments are conducted on a recent HRI database with available ground-truth [3] featuring natural human robot interactions.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Leveraging the robot dialog state for visual focus of attention recognition

Sheikhi

Khalidov

Klotz

et al. 2013

Proceedings of the 15th ACM on International Conference on Multimodal Interaction

Self Cite

View full text Add to dashboard Cite

The Visual Focus of Attention (what or whom a person is looking at) or VFOA is a fundamental cue in non-verbal communication and plays an important role when designing effective human-machine interaction systems. However, recognizing the VFOA of an interacting person is difficult for a robot, since due to low resolution imaging, eye gaze estimation is not possible. Rather, head pose cue is used as a substitute for gaze, but leads to ambiguities in its interpretation as VFOA indicator. In this paper, we investigate the use of the robot conversational state, which the robot is aware of, as contextual information to improve VFOA recognition from head pose. We propose a dynamic Bayesian model that accounts for the robot state (speaking status, person he addresses, reference to objects) along with a dynamic head-togaze mapping function. Experiments on a publicly available human-robot interaction dataset, where a humanoid robot plays the role of an art guide and quiz master, shows that using such conversational context is effective in improving VFOA.

show abstract

Combining dynamic head pose–gaze mapping with the robot conversational state for attention recognition in human–robot interactions

Sheikhi

Odobez

2015

Pattern Recognition Letters

Self Cite

View full text Add to dashboard Cite

The ability to recognize the Visual Focus of Attention (VFOA, i.e. what or whom a person is looking at) of people is important for robots or conversational agents interacting with multiple people, since it plays a key role in turn-taking, engagement or intention monitoring. As eye gaze estimation is often impossible to achieve, most systems currently rely on head pose as an approximation, creating ambiguities since the same head pose can be used to look at different VFOA targets. To address this challenge, we propose a dynamic Bayesian model for the VFOA recognition from head pose, where we make two main contributions. First, taking inspiration from behavioral models describing the relationships between the body, head and gaze orientations involved in gaze shifts, we propose novel gaze models that dynamically and more accurately predict the expected head orientation used for looking in a given gaze target direction. This is a neglected aspect of previous works but essential for recognition. Secondly, we propose to exploit the robot conversational state (when he speaks, objects to which he refers) as context to set appropriate priors on candidate VFOA targets and reduce the inherent VFOA ambiguities. Experiments on a public dataset where the humanoid robot NAO plays the role of an art guide and quiz master demonstrate the benefit of the two contributions.

show abstract

Investigating the midline effect for visual focus of attention recognition

Cited by 9 publications

References 11 publications

Entropy-weighted feature-fusion method for head-pose estimation

Entropy-weighted feature-fusion method for head-pose estimation

Leveraging the robot dialog state for visual focus of attention recognition

Combining dynamic head pose–gaze mapping with the robot conversational state for attention recognition in human–robot interactions

Contact Info

Product

Resources

About