3DOF Pedestrian Trajectory Prediction Learned from Long-Term Autonomous Mobile Robot Deployment Data

Abstract

This paper presents a novel 3DOF pedestrian trajectory prediction approach for autonomous mobile service robots. While most previously reported methods are based on learning of 2D positions in monocular camera images, our approach uses range-finder sensors to learn and predict 3DOF pose trajectories (i.e. 2D position plus 1D rotation within the world coordinate system). Our approach, T-Pose-LSTM (Temporal 3DOF-Pose Long-Short-Term Memory), is trained using long-term data from real-world robot deployments and aims to learn context-dependent (environment-and timespecific) human activities. Our approach incorporates long-term temporal information (i.e. date and time) with short-term pose observations as input. A sequence-to-sequence LSTM encoderdecoder is trained, which encodes observations into LSTM and then decodes as predictions. For deployment, it can perform on-the-fly prediction in real-time. Instead of using manually annotated data, we rely on a robust human detection, tracking and SLAM system, providing us with examples in a global coordinate system. We validate the approach using more than 15K pedestrian trajectories recorded in a care home environment over a period of three months. The experiment shows that the proposed T-Pose-LSTM model advances the state-of-the-art 2D-based method for human trajectory prediction in long-term mobile robot deployments.

Time-varying Pedestrian Flow Models for Service Robots

“…That assumes a reactive approach, where a robot estimates the people velocities by tracking them and then replans its trajectory. As reported in [10], the errors of state-of-the-art methods exceed 0.4 m for prediction horizons of 1 s, which means that reactive navigation around people still requires a high-speed sense-predict-plan-act loop.…”

“…To overcome the limitations of reactive approaches, a robot could learn natural motion patterns from long-term experience [11], [10], and plan its path while anticipating people walking in learned directions even if it does perceive any humans at a given moment. In other words, knowledge of the typical patterns of people movement could improve socially-compliant navigation by planning robot trajectories so that robots would follow the natural flows of people, and avoid congestions and areas where they would cause a nuisance.…”

Multisensor Online Transfer Learning for 3D LiDAR-Based Human Detection with a Mobile Robot

“…The 3D LiDAR-based cluster detector and the human classifier are originally from our recent work [12], while the former has been incorporated in different problems [23], [24]. As input of this module, a 3D LiDAR scan is first properly segmented into different clusters using an adaptive clustering approach.…”

Forecasting People Trajectories and Head Poses by Jointly Reasoning on Tracklets and Vislets

“…Forecasting trajectories from images, however, is a complex problem and, probably for this reason, it has only recently emerged as a popular computer vision research topic. In particular, the modern re-visitation of Long Short Term Memory (LSTM) architectures [41], has enabled a leap forward in performance [31], [34], [72], [73], [78]. On one side, LSTM has allowed a seamless encoding of the social interplay among pedestrians [3], [31].…”