A Multi-Modal Person Perception Framework for Socially Interactive Mobile Service Robots

Müller, Steffen; Wengefeld, Tim; Trinh, Thanh Q.; Aganian, Dustin; Eisenbach, Markus; Groß, Horst–Michael

doi:10.3390/s20030722

Cited by 17 publications

(7 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Other works also address the problem of fusing data from sensors, such as 2D laser scanners or RGBD cameras, to socially navigate. The work in [38] proposed data fusion from a fish-eye camera, a RGBD camera and a 2D laser scanner for people modeling and tracking. Additionally, face descriptors are extracted to re-identify users.…”

Section: Discussionmentioning

confidence: 99%

Sensor Fusion for Social Navigation on a Mobile Robot Based on Fast Marching Square and Gaussian Mixture Model

Mora

Prados

Méndez

et al. 2022

Sensors

View full text Add to dashboard Cite

Mobile robot navigation has been studied for a long time, and it is nowadays widely used in multiple applications. However, it is traditionally focused on two-dimensional geometric characteristics of the environments. There are situations in which robots need to share space with people, so additional aspects, such as social distancing, need to be considered. In this work, an approach for social navigation is presented. A multi-layer model of the environment containing geometric and topological characteristics is built based on the fusion of multiple sensor information. This is later used for navigating the environment considering social distancing from individuals and groups of people. The main novelty is combining fast marching square for path planning and navigation with Gaussian models to represent people. This combination allows to create a continuous representation of the environment from which smooth paths can be extracted and modified according to dynamically captured data. Results prove the practical application of the method on an assistive robot for navigating indoor scenarios, including a behavior for crossing narrow passages. People are efficiently detected and modeled to assure their comfort when robots are around.

show abstract

Section: Discussionmentioning

confidence: 99%

Sensor Fusion for Social Navigation on a Mobile Robot Based on Fast Marching Square and Gaussian Mixture Model

Mora

Prados

Méndez

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…Thus, these configurations should not be considered for robotics applications. Furthermore, at late positions (9)(10)(11)(12)(13)(14)(15)(16)(17) only AGW [35] can cope with the inference speed of channel-wise attention blocks to some degree. Also at middle positions (5-8) channel-wise attention is faster to compute than non-local attention.…”

Section: B Analysis Of Inference Speed Of Attention Blocks For Person...mentioning

confidence: 99%

“…occasionally out of sight for some time. Therefore, person re-identification (re-id) plays a key role in such scenarios [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16] either to provide robust features that can be used internally in a tracker for track continuation in case of temporal total occlusions [17] or to continue a track by re-identifying a person after a track is lost in a separate module [18].…”

Section: Introductionmentioning

confidence: 99%

“…In recent years, ConvNet-based features have proven to be very reliable for non-biometric re-identification after the user has been unobserved for some time. Therefore, these features can be used as visual cues in a probabilistic multi-person tracking framework [17]. A person detector extracts cropped RGB images for all persons in the observed scene.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Little Bit Attention Is All You Need for Person Re-Identification

Eisenbach¹,

Lübberstedt²,

Aganian³

et al. 2023

Preprint

View full text Add to dashboard Cite

Person re-identification plays a key role in applications where a mobile robot needs to track its users over a long period of time, even if they are partially unobserved for some time, in order to follow them or be available on demand. In this context, deep-learning based real-time feature extraction on a mobile robot is often performed on specialpurpose devices whose computational resources are shared for multiple tasks. Therefore, the inference speed has to be taken into account. In contrast, person re-identification is often improved by architectural changes that come at the cost of significantly slowing down inference. Attention blocks are one such example. We will show that some well-performing attention blocks used in the state of the art are subject to inference costs that are far too high to justify their use for mobile robotic applications. As a consequence, we propose an attention block that only slightly affects the inference speed while keeping up with much deeper networks or more complex attention blocks in terms of re-identification accuracy. We perform extensive neural architecture search to derive rules at which locations this attention block should be integrated into the architecture in order to achieve the best trade-off between speed and accuracy. Finally, we confirm that the best performing configuration on a re-identification benchmark also performs well on an indoor robotic dataset.

show abstract

“…The research platform uses a complete multimodal person tracking framework [ 31 ], which is also capable of re-identifying the patient among other detected persons. As detection modules, OpenPose [ 32 ], YOLO ( , accessed on 13 September 2021), and a laser-based detector are able to detect persons’ legs, even when mobility aids [ 33 ], i.e., crutches, walkers, or wheelchairs, are used.…”

Section: Mobile Gait Self-training Under Real Clinical Environment Conditionsmentioning

confidence: 99%

Robot-Assisted Gait Self-Training: Assessing the Level Achieved

Scheidig

Schütz

Trinh

et al. 2021

Sensors

Self Cite

View full text Add to dashboard Cite

This paper presents the technological status of robot-assisted gait self-training under real clinical environment conditions. A successful rehabilitation after surgery in hip endoprosthetics comprises self-training of the lessons taught by physiotherapists. While doing this, immediate feedback to the patient about deviations from the expected physiological gait pattern during training is important. Hence, the Socially Assistive Robot (SAR) developed for this type of training employs task-specific, user-centered navigation and autonomous, real-time gait feature classification techniques to enrich the self-training through companionship and timely corrective feedback. The evaluation of the system took place during user tests in a hospital from the point of view of technical benchmarking, considering the therapists’ and patients’ point of view with regard to training motivation and from the point of view of initial findings on medical efficacy as a prerequisite from an economic perspective. In this paper, the following research questions were primarily considered: Does the level of technology achieved enable autonomous use in everyday clinical practice? Has the gait pattern of patients who used additional robot-assisted gait self-training for several days been changed or improved compared to patients without this training? How does the use of a SAR-based self-training robot affect the motivation of the patients?

show abstract

A Multi-Modal Person Perception Framework for Socially Interactive Mobile Service Robots

Cited by 17 publications

References 27 publications

Sensor Fusion for Social Navigation on a Mobile Robot Based on Fast Marching Square and Gaussian Mixture Model

Sensor Fusion for Social Navigation on a Mobile Robot Based on Fast Marching Square and Gaussian Mixture Model

A Little Bit Attention Is All You Need for Person Re-Identification

Robot-Assisted Gait Self-Training: Assessing the Level Achieved

Contact Info

Product

Resources

About