Using Panoramic Videos for Multi-Person Localization and Tracking In A 3D Panoramic Coordinate

Yang, Fan; Li, Feiran; Wu, Yang; Sakti, Sakriani; Nakamura, Satoshi

doi:10.1109/icassp40776.2020.9053497

Cited by 4 publications

(4 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The evaluation was performed offline on the pre-recorded test set with a PC equipped with an NVIDIA GTX 1650 GPU, 16 GB of RAM, and an Intel i7-4790 CPU. We also compared the performance of our proposal with the Multi-person Panoramic Localization and Tracking (MPLT) [26] introduced in Sec. 2.2.…”

Section: Methodsmentioning

confidence: 99%

“…Recent studies used a couple of calibrated omnidirectional cameras to achieve accurate 3D people tracking in panoramic videos [25]. The proposal of Yang et al [26], called Multi-person Panoramic Localization and Tracking (MPLT), instead relies on four calibrated perspective cameras, positioned such that they cover a field of view of 360 • to simulate an omnidirectional camera. The OpenPifPaf framework [13] is used for skeleton detection and a pinhole camera model is used to map detections in the 3D space, under the assumption that the height of a person is almost constant.…”

Section: People Trackingmentioning

confidence: 99%

See 1 more Smart Citation

People Tracking in Panoramic Video for Guiding Robots

Bacchin¹,

Berno²,

Menegatti³

et al. 2022

Preprint

View full text Add to dashboard Cite

A guiding robot aims to effectively bring people to and from specific places within environments that are possibly unknown to them. During this operation the robot should be able to detect and track the accompanied person, trying never to lose sight of her/him. A solution to minimize this event is to use an omnidirectional camera: its 360 • Field of View (FoV) guarantees that any framed object cannot leave the FoV if not occluded or very far from the sensor. However, the acquired panoramic videos introduce new challenges in perception tasks such as people detection and tracking, including the large size of the images to be processed, the distortion effects introduced by the cylindrical projection and the periodic nature of panoramic images. In this paper, we propose a set of targeted methods that allow to effectively adapt to panoramic videos a standard people detection and tracking pipeline originally designed for perspective cameras. Our methods have been implemented and tested inside a deep learning-based people detection and tracking framework with a commercial 360 • camera. Experiments performed on datasets specifically acquired for guiding robot applications and on a real service robot show the effectiveness of the proposed approach over other state-of-the-art systems. We release with this paper the acquired and annotated datasets and the open-source implementation of our method.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: People Trackingmentioning

confidence: 99%

People Tracking in Panoramic Video for Guiding Robots

Bacchin¹,

Berno²,

Menegatti³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In Yang et al [16], a multi-person panoramic localization and tracking (MPLT) dataset is presented to enable model evaluation for 3D panoramic multi-person localization and tracking. It represents real-world scenarios and contains a crowd of people in each frame.…”

Section: • Videos Datasetsmentioning

confidence: 99%

A Dataset of Annotated Omnidirectional Videos for Distancing Applications

et al. 2021

View full text Add to dashboard Cite

Omnidirectional (or 360°) cameras are acquisition devices that, in the next few years, could have a big impact on video surveillance applications, research, and industry, as they can record a spherical view of a whole environment from every perspective. This paper presents two new contributions to the research community: the CVIP360 dataset, an annotated dataset of 360° videos for distancing applications, and a new method to estimate the distances of objects in a scene from a single 360° image. The CVIP360 dataset includes 16 videos acquired outdoors and indoors, annotated by adding information about the pedestrians in the scene (bounding boxes) and the distances to the camera of some points in the 3D world by using markers at fixed and known intervals. The proposed distance estimation algorithm is based on geometry facts regarding the acquisition process of the omnidirectional device, and is uncalibrated in practice: the only required parameter is the camera height. The proposed algorithm was tested on the CVIP360 dataset, and empirical results demonstrate that the estimation error is negligible for distancing applications.

show abstract

“…Some researchers showed that by using post-processing, 360-degree video streaming can support functions other than only streaming videos. Rhee showed that 3D virtual objects can be seamlessly composited on 360-degree streaming videos [13], and Yang et al [32] presented 360-degree videos that can generate person localization data by proposing a multi-person localization and tracking algorithm for 360-degree videos. We adopted the 360 technique from the aforementioned studies because it can support a sophisticated monitoring function from a micro-perspective.…”

Section: Micro Perspective Of Udt Using 360 Techniquesmentioning

confidence: 99%

GeoACT: Augmented Control Tower using Virtual and Real Geospatial Data

Ssin¹,

Cho²,

Woo³

2021

IxD&A

View full text Add to dashboard Cite

Despite prior efforts to solve urban problems and build smart cities using large-scale urban information, researchers have yet to develop tools for city administrators to easily access a diverse mix of city information. To address this limitation, this article proposes an augmented reality visualization system, GeoACT, which can develop a digital twin to present city information from both macro and micro perspectives. GeoACT employs a 3D virtual city miniature with real and virtual IoT information to provide a macro perspective view, and a 360-degree-video-based visualization technique is used for a micro perspective view. We applied GeoACT to an urban space and demonstrated that it could effectively visualize city information through an augmented reality Head Mounted Display (HMD). Furthermore, GeoACT can be used in urban control centers.

show abstract

Using Panoramic Videos for Multi-Person Localization and Tracking In A 3D Panoramic Coordinate

Cited by 4 publications

References 25 publications

People Tracking in Panoramic Video for Guiding Robots

People Tracking in Panoramic Video for Guiding Robots

A Dataset of Annotated Omnidirectional Videos for Distancing Applications

GeoACT: Augmented Control Tower using Virtual and Real Geospatial Data

Contact Info

Product

Resources

About