Egocentric Height Estimation

Abstract

Egocentric, or first-person vision which became popular in recent years with an emerge in wearable technology, is different than exocentric (third-person) vision in some distinguishable ways, one of which being that the camera wearer is generally not visible in the video frames. Recent work has been done on action and object recognition in egocentric videos, as well as work on biometric extraction from first-person videos. Height estimation can be a useful feature for both soft-biometrics and object tracking. Here, we propose a method of estimating the height of an egocentric camera without any calibration or reference points. We used both traditional computer vision approaches and deep learning in order to determine the visual cues that results in best height estimation. Here, we introduce a framework inspired by two stream networks comprising of two Convolutional Neural Networks, one based on spatial information, and one based on information given by optical flow in a frame. Given an egocentric video as an input to the framework, our model yields a height estimate as an output. We also incorporate late fusion to learn a combination of temporal and spatial cues. Comparing our model with other methods we used as baselines, we achieve height estimates for videos with a Mean Average Error of 14.04 cm over a range of 103 cm of data, and classification accuracy for relative height (tall, medium or short) up to 93.75% where chance level is 33%.

Recognizing Camera Wearer from Hand Gestures in Egocentric Videos

“…They computed block-wise optical flows from the given egocentric video and trained a small CNN for camera wearer classification. Finocchiaro et al [6] estimated the height of the camera from the ground using only the egocentric video without any intrinsic camera information. They have used a 2stream CNN based regression model, which regresses the height of the camera wearer from the given input RGB video and its derived optical flows.…”

Privacy as a Feature for Body-Worn Cameras [In the Spotlight]

“…Primary and secondary information captured by BWCs can be used to comprehensively describe the behaviors of the wearer. Such a description may be employed to profile wearers through their physical characteristics [8], [9], their activities [10], the foods they eat [11], and how they interact with others [12]. Each additional day of BWC usage marks the collation of more information, also enabling the inference of insights that are not directly observable.…”