Human activity analysis: a personal robot integrating a framework for robust person detection and tracking and physical based motion analysis

Abstract

The analysis of certain parameters related to cognitive and to motor humans’ activities in everyday life conditions can allow to detect potential behavioral troubles, make diagnoses and assess patients’ progress after a therapy. Within this context, personal robots can provide an autonomous movable platform for embedded sensors allowing to detect and track humans while ensuring an optimal observability of the person’s activity in complex and cluttered environments. This paper presents a framework combining a multimodal human detector based on sensors embedded in a mobile robot and a decisional engine exploiting the fuzzy logic mechanisms to make the robot track humans, maximizing observability and facing losses of detection. The robustness of this framework is evaluated experimentally in home spaces through different scenarios. Such a mobile system provides an effective marker-less motion capture means for sensing human activity in non-invasive fashion. We present a physical model based method exploiting the features of the system and of the embedded Kinect. Its performances are evaluated first comparing the results to those obtained with a precise 3D motion capture marker based system and to data obtained from a dynamic posturography platform. Then an experiment in real life conditions is performed to assess the system sensitivity to some gait disturbances.

Human Activity-Understanding: A Multilayer Approach Combining Body Movements and Contextual Descriptors Analysis

“…In this software, the anthropomorphic model is rescaled automatically to match the height and weight of the subject's morphology. The use of a model-based approach with the simulator filters the input data by forcing the respect of the kinematics constraints [35]. A model-based approach is employed to replay the captured kinematics data in a kinematically- and dynamically-consistent manner, in order to extract accurate kinematics indicators from the human motion and to align the perceived movements with physical constraints acting on the human body.…”

“…This allows for an in-depth analysis in non-controlled environmental conditions, without any environmental equipment required. In a previous work [35], we presented a framework combining a multimodal human detector based on sensors embedded in a mobile robot, and a decisional engine exploiting fuzzy logic mechanisms to enable the robot to track humans, maximizing observability and confronting losses of detection. The idea that motivated this work was to exploit the ability of the robot to track humans and maximize the observability of their movements in order to understand their behaviour.…”

“…In the following sections, we present a new method to achieve the challenging task of human activity recognition. Here, the assessment of this method is performed on data collected with a fixed Kinect sensor [1], but the same results could be obtained with data collected from a Kinect embedded on the robot while tracking the person as introduced in [35].…”

“…This replay is divided into two successive steps, with an initial kinematic filter followed by a dynamic simulation of the model. Since the segments of the skeleton from OpenNI have no constant lengths, the joint data must first be filtered in order to be consistent with the kinematics of the human model, thus respecting the joint limits, collisions and other kinematics constraints [35]. An exclusively kinematic simulation is performed with the human model, tracking the joint positions in space from the skeleton data through proportional-derivative coupling between selected segments of the model and the captured picking objects cleaning objects taking food arranging objects having meal Figure 2.…”

Point-and-Command Paradigm for Interaction with Assistive Robots

“…In the section relating to the robot's decision making, fuzzy logic (FL) was used when information was vague, inconsistent or incomplete (decisions under uncertainties), e.g., for person detection, and tracking in cluttered and unstructured environments [31]. Other stochastic models, like Markov decision processes (MDP) [32], partially observable Markov decision processes (POMDP) [33], Bayesian networks (BN) [34], deep dynamic Bayesian networks (DDBN) [35], and hidden Markov models (HMM) [36], have been used in designing user-assistive systems for intention recognition or speech recognition.…”