People detection and tracking using RGB-D cameras for mobile robots

Abstract

People detection and tracking is an essential capability for mobile robots in order to achieve natural human-robot interaction. In this article, a human detection and tracking system is designed and validated for mobile robots using color data with depth information RGB-depth (RGB-D) cameras. The whole framework is composed of human detection, tracking and re-identification. Firstly, ground points and ceiling planes are removed to reduce computation effort. A prior-knowledge guided random sample consensus fitting algorithm is used to detect the ground plane and ceiling points. All left points are projected onto the ground plane and subclusters are segmented for candidate detection. Meanshift clustering with an Epanechnikov kernel is conducted to partition different points into subclusters. We propose the new idea of spatial region of interest plan view maps which are employed to identify human candidates from point cloud subclusters. Here, a depthweighted histogram is extracted online to feature a human candidate. Then, a particle filter algorithm is adopted to track the human's motion. The integration of the depth-weighted histogram and particle filter provides a precise tool to track the motion of human objects. Finally, data association is set up to re-identify humans who are tracked. Extensive experiments are conducted to demonstrate the effectiveness and robustness of our human detection and tracking system.

Soft labeling with quasi-Gaussian structure for training samples of deep classification trackers

“…Since the IoU value of the sample is negatively correlated to the center distance between the sample and the searching region in SiamFC, we set this relationship as I i ¼ ÀbR i þ 1, where R i and b > 0 denotes the center distance and negative correlation coefficient, respectively. Then the soft labeling of SiamFC-label is expressed as equation (5) and its visualization is shown intuitively in Figure 7…”

“…42 SiamFC with three scales is selected as baseline tracker since this version runs faster than the one with five scales and only performs slightly lower. We set the parameters of soft labeling with quasi-Gaussian structure in equation (5) as Table 1. The means of Gaussian distribution are set their values as 1 to satisfy the first constraint in equation (4), which makes the response values of samples belonging to the same class are no longer the same but positively correlated with their IoU value.…”

“…Visual object tracking, one of the most important tasks in many robot applications, has been widely used in many fields such as intelligent manufacturing, human–computer interaction, video surveillance, and robotics. 1 –3 It is an indispensable part of robots 4,5 serving as the “eye” for robots to communicate with the world as the Figure 1 shown.…”

3-D Refuse-Derived Fuel Particle Tracking-by-Detection Using a Plenoptic Camera System

“…In case that scenes can be captured three-dimensionally, such as using stereo cameras, RGB-D cameras, ToF cameras, or plenoptic cameras, clustering methods can be applied to the obtained point clouds to implement object detection. A new idea using mean shift clustering candidate segmentation for people detection based on the point clouds gathered by an RGB-D camera is introduced in [17]. The authors of [18] propose a fast clustering method on the basis of densitybased spatial clustering of applications with noise (DBSCAN) algorithm [19] to realize traffic detection for self-driving technology.…”

Astronaut Visual Tracking of Flying Assistant Robot in Space Station Based on Deep Learning and Probabilistic Model

“…Alone [28][29][30][31] or combined with the traditional methods [32], these novel deep learning methods are more and more widely used in people detection research. Based on the detection results, Kalman filter [33][34][35], particle filter [17,36], and probabilistic model [32] are often used to accomplish accurate and consecutive people tracking task.…”