Detecting and identifying people in mobile videos

Abstract

In this paper, we propose a system capable of detecting and identifying people in videos captured by smart phones. We discuss the challenges to extend existing location aware multimedia applications from annotating static landmarks in distance to annotating dynamic people in close range with significant pose variations. We propose to use a hybrid video and RF tracking system to enable accurate observer and target localization, and extract part models comprised of Maximally Stable Color Regions for each target. The model can efficiently detect possible positions of targets in the video, which are then used as dynamic landmarks to calibrate the camera orientation. Finally, positions of all targets in the video are jointly estimated using both visual features and spatial constraints. Experiments show that our approach can locate identified targets in video with significantly higher accuracy than back-projection using camera orientation estimations from accelerometers and magnetometers.