Fast and Robust 3D Person Detector and Posture Estimator for Mobile Robotic Applications

Lewandowski, Benjamin; Liebner, Jonathan; Wengefeld, Tim; Müller, Steffen; Groß, Horst–Michael

doi:10.1109/icra.2019.8793712

Cited by 22 publications

(10 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The most significant detector in our setup is OpenPose [23], a 2D skeleton detector operating on the images of the three fisheye cameras. According to [24], OpenPose outperformed several image-based detectors with respect to detection quality. OpenPose is a CNN-based approach and is only real-time capable if running on a GPU.…”

Section: Detection Modulesmentioning

confidence: 99%

“…Due to the wide opening angle and the low resolution of the fisheye images, the range of this detector is limited to about 5m. In order to cover distances of up to 10m and for generating point cloud segments for further analysis of body orientation and posture analysis, the clustering method from [24] is used on the point cloud extracted from the Kinect2 sensor data.…”

Section: Detection Modulesmentioning

confidence: 99%

“…Besides the position, the orientations of persons to be tracked and their postures (standing, sitting, squatting) are important information for making decisions during HRI. Therefore, the detected point cloud segments are further processed by a CNN to classify posture [24] and upper body orientation [27]. The observations of these properties are modeled as discrete distributions with three bins for the posture classes and eight bins for the orientation.…”

Section: Posture and Orientationmentioning

confidence: 99%

“…That study shows that a tracking brings a real benefit over just taking the frame wise detection results. Also for results of the posture estimation the interested reader is referred to a separate study [24].…”

Section: Benchmark On Labeled Datasetmentioning

confidence: 99%

See 3 more Smart Citations

A Multi-Modal Person Perception Framework for Socially Interactive Mobile Service Robots

Müller

Wengefeld

Trinh

et al. 2020

Sensors

Self Cite

View full text Add to dashboard Cite

In order to meet the increasing demands of mobile service robot applications, a dedicated perception module is an essential requirement for the interaction with users in real-world scenarios. In particular, multi sensor fusion and human re-identification are recognized as active research fronts. Through this paper we contribute to the topic and present a modular detection and tracking system that models position and additional properties of persons in the surroundings of a mobile robot. The proposed system introduces a probability-based data association method that besides the position can incorporate face and color-based appearance features in order to realize a re-identification of persons when tracking gets interrupted. The system combines the results of various state-of-the-art image-based detection systems for person recognition, person identification and attribute estimation. This allows a stable estimate of a mobile robot’s user, even in complex, cluttered environments with long-lasting occlusions. In our benchmark, we introduce a new measure for tracking consistency and show the improvements when face and appearance-based re-identification are combined. The tracking system was applied in a real world application with a mobile rehabilitation assistant robot in a public hospital. The estimated states of persons are used for the user-centered navigation behaviors, e.g., guiding or approaching a person, but also for realizing a socially acceptable navigation in public environments.

show abstract

Section: Detection Modulesmentioning

confidence: 99%

Section: Detection Modulesmentioning

confidence: 99%

Section: Posture and Orientationmentioning

confidence: 99%

Section: Benchmark On Labeled Datasetmentioning

confidence: 99%

See 2 more Smart Citations

A Multi-Modal Person Perception Framework for Socially Interactive Mobile Service Robots

Müller

Wengefeld

Trinh

et al. 2020

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…different types of clothes and hairstyles. This approach, combined with the feature-based detection method from Lewandowski et al (2019), was used in Wengefeld et al (2019) to estimate human poses based on performant features extracted from colored point clouds. The detection method used a layer-based approach to calculate feature descriptors for each layer in a point cluster and concatenated the histograms to form feature vectors.…”

Section: Introductionmentioning

confidence: 99%

CNN-based People Detection in Voxel Space using Intensity Measurements and Point Cluster Flattening

Dybedal¹,

Hovland²

2021

MIC

View full text Add to dashboard Cite

In this paper real-time people detection is demonstrated in a relatively large indoor industrial robot cell as well as in an outdoor environment. Six depth sensors mounted at the ceiling are used to generate a merged point cloud of the cell. The merged point cloud is segmented into clusters and flattened into gray-scale 2D images in the xy and xz planes. These images are then used as input to a classifier based on convolutional neural networks (CNNs). The final output is the 3D position (x, y, z) and bounding box representing the human. The system is able to detect and track multiple humans in real-time, both indoors and outdoors. The positional accuracy of the proposed method has been verified against several ground truth positions, and was found to be within the point-cloud voxel-size used, i.e. 0.04 m. Tests on outdoor datasets yielded a detection recall of 76.9 % and an F1 score of 0.87.

show abstract

Now I Need Help! Passing Doors and Using Elevators as an Assistance Requiring Robot

2019

View full text Add to dashboard Cite

Fast and Robust 3D Person Detector and Posture Estimator for Mobile Robotic Applications

Cited by 22 publications

References 23 publications

A Multi-Modal Person Perception Framework for Socially Interactive Mobile Service Robots

A Multi-Modal Person Perception Framework for Socially Interactive Mobile Service Robots

CNN-based People Detection in Voxel Space using Intensity Measurements and Point Cluster Flattening

Now I Need Help! Passing Doors and Using Elevators as an Assistance Requiring Robot

Contact Info

Product

Resources

About