CamLoc: Pedestrian Location Estimation through Body Pose Estimation on Smart Cameras

Abstract

Recent advancements in energy-efficient hardware technology is driving the exponential growth we are experiencing in the Internet of Things (IoT) space, with more pervasive computations being performed near to data generation sources. A range of intelligent devices and applications performing local detection is emerging (activity recognition, fitness monitoring, etc.) bringing with them obvious advantages such as reducing detection latency for improved interaction with devices and safeguarding user data by not leaving the device. Video processing holds utility for many emerging applications and data labelling in the IoT space. However, performing this video processing with deep neural networks at the edge of the Internet is not trivial. In this paper we show that pedestrian location estimation using deep neural networks is achievable on fixed cameras with limited compute resources. Our approach uses pose estimation from key body points detection to extend pedestrian skeleton when whole body not in image (occluded by obstacles or partially outside of frame), which achieves better location estimation performance (infrence time and memory footprint) compared to fitting a bounding box over pedestrian and scaling. We collect a sizable dataset comprising of over 2100 frames in videos from one and two surveillance cameras pointing from different angles at the scene, and annotate each frame with the exact position of person in image, in 42 different scenarios of activity and occlusion. We compare our pose estimation based location detection with a popular detection algorithm, YOLOv2, for overlapping bounding-box generation, our solution achieving faster inference time (15x speedup) at half the memory footprint, within resource capabilities on embedded devices, which demonstrate that CamLoc is an efficient solution for location estimation in videos on smart-cameras.

Sensor-fusion Location Tracking System using Hybrid Multimodal Deep Neural Network

“…In the future, this data collection can be automated, either by robots roaming the indoor space to update WiFi radio maps or by mass unlabelled data collection from users roaming naturally in the environment. With labelling solutions based on computer vision [22], this data can be used fruitfully.…”

“…We aim to customise an end-to-end multimodal deep neural network for the indoor localization task to produce location estimation based on inertial sensors and WiFi fingerprints data. Training directly on data has its drawbacks, that of moving the challenges to the quality of training dataset and cross-sensor modality alignment, although this can be eventually automated by other systems such as vision-based systems [22].…”

Calibrating Recurrent Neural Networks on Smartphone Inertial Sensors for Location Tracking

“…However, with the increasing availability of data, which is hard to model entirely with precise mathematical formulation, deep learning adoption offers the benefit of extracting complex features automatically from data. While we move the complexity of modeling to generating good quality labeled data for training, we believe this is achievable with ingenious solution to facilitate data collection and labeling such as using camera infrastructure opportunistically [7].…”

“…Although we move the burden of developing localization systems to generating good labeled training sets, we believe this is more scalable since data collection is easier than human intervention to alter previous systems for new environments and edge cases. Solutions based on infrastructure cameras to extract location estimation [7] for sensor data labeling can be one approach to enhance training data collection at scale.…”

Alternatives for Locating People Using Cameras and Embedded AI Accelerators: A Practical Approach

“…The most common alternatives for person detection using convolutional neural networks (CNNs) are object detectors, which provide bounding boxes of the persons, and pose estimation, which provides the position of the different key body joints of each person. Cosma et al [2] compared these two methods, obtaining better results with pose detection networks, which are more resistant to occlusions. Moreover, the correct processing of the detected pose allows for estimation of the position of the person's feet more accurately even when they do not appear in the image.…”