Understanding of Human Behavior with a Robotic Agent Through Daily Activity Analysis

Abstract

Personal assistive robots to be realized in the near future should have the ability to seamlessly coexist with humans in unconstrained environments, with the robot's capability to understand and interpret the human behavior during human-robot cohabitation significantly contributing towards this end. Still, the understanding of human behavior through a robot is a challenging task as it necessitates a comprehensive representation of the high-level structure of the human's behavior from the robot's low-level sensory input. The paper at hand tackles this problem by demonstrating a robotic agent capable of apprehending human daily activities through a method, the Interaction Unit analysis, that enables activities' decomposition into a sequence of units, each one associated with a behavioral factor. The modelling of human behavior is addressed with a Dynamic Bayesian Network that operates on top of the Interaction Unit, offering quantification of the behavioral factors and the formulation of the human's behavioral model. In addition, lightweight human action and object manipulation monitoring strategies have been developed, based on RGB-D and laser sensors, tailored for onboard robot operation. As a proof of concept, we used our robot to evaluate the ability of the method to differentiate among the examined human activities, as well as to assess the capability of behavior modeling of people with Mild Cognitive Impairment. Moreover, we deployed our robot in 12 real

Temporal action detection based on two-stream You Only Look Once network for elderly care service robot

“…To achieve the robot's capability to understand and interpret human behavior during human-robot cohabitation, a lot of attempts had been made. Kostavelis et al 20 proposed a method named the Interaction Unit analysis, which modeled human behaviors based on a Dynamic Bayesian Network, and tried to tackle the comprehensive representation of the high-level structure of human's behavior for the robot's low-level sensory input. Ramrez-Amaro et al 21 presented semantic-based methods for understanding human movements in robotic applications.…”

A Review: Point Cloud-Based 3D Human Joints Estimation

“…Wang et al [ 113 , 114 ] established a model as a bridge between the input point cloud of the human body and the robot, so as to achieve the purpose of human–robot motion retargeting. The activity was decomposed into multiple unit sequences, each unit was related to an important factor of behavior [ 115 ], and then was inputted into a dynamic Bayesian network to analyze human behavior intentions and realize human–computer interaction.…”

Prognosing Human Activity Using Actions Forecast and Structured Database

“…The label space concept allows also the motion recognition. Recently the authors of [30] have described the approach dealing with understanding human daily activities through the so-called Interaction Unit analysis that enables decomposition of activities into a sequence of units. Each of these units is associated with a behavioural factor.…”