Detecting Engagement in Egocentric Video

Su, Yu-Chuan; Grauman, Kristen

doi:10.1007/978-3-319-46454-1_28

Cited by 45 publications

(37 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Aghaei et al [1] released the EgoSocialStyle dataset of 130k images from eight users, for the purpose of characterising the social patterns of a person relying exclusively on visual data. UT EE [37] gathered and crowd source annotated 27 videos from nine users, covering 14 hours in order to detecting the wearer's engagement with the surrounding environment. An event segmentation dataset was published by Gupta and Gurrin [12], consisting of 14k images from ten participants.…”

Section: Personal Data Test Collectionsmentioning

confidence: 99%

Rethinking the Test Collection Methodology for Personal Self-tracking Data

Hopfgartner

Gurrin

Joho

2019

MultiMedia Modeling

View full text Add to dashboard Cite

While vast volumes of personal data are being gathered daily by individuals, the MMM community has not really been tackling the challenge of developing novel retrieval algorithms for this data, due to the challenges of getting access to the data in the first place. While initial efforts have taken place on a small scale, it is our conjecture that a new evaluation paradigm is required in order to make progress in analysing, modeling and retrieving from personal data archives. In this position paper, we propose a new model of Evaluation-as-a-Service that re-imagines the test collection methodology for personal multimedia data in order to address the many challenges of releasing test collections of personal multimedia data.

show abstract

Section: Personal Data Test Collectionsmentioning

confidence: 99%

Rethinking the Test Collection Methodology for Personal Self-tracking Data

Hopfgartner

Gurrin

Joho

2019

MultiMedia Modeling

View full text Add to dashboard Cite

show abstract

“…This method [30] used low temporal resolution image sequences to detect the social interactions. Su et al [50] proposed a video summarization approach to detect the engagement using long-term ego-motion cues (i.e., gaze). This approach [50] consists of three stages that are frame prediction, interval prediction, and classification with the trained model.…”

Section: Performance Comparison Of the Proposed Framework Withmentioning

confidence: 99%

Egocentric Video Summarization Based on People Interaction Using Deep Learning

Ghafoor

Javed

Irtaza

et al. 2018

Mathematical Problems in Engineering

View full text Add to dashboard Cite

The availability of wearable cameras in the consumer market has motivated the users to record their daily life activities and post them on the social media. This exponential growth of egocentric videos demand to develop automated techniques to effectively summarizes the first-person video data. Egocentric videos are commonly used to record lifelogs these days due to the availability of low cost wearable cameras. However, egocentric videos are challenging to process due to the fact that placement of camera results in a video which presents great deal of variation in object appearance, illumination conditions, and movement. This paper presents an egocentric video summarization framework based on detecting important people in the video. The proposed method generates a compact summary of egocentric videos that contains information of the people whom the camera wearer interacts with. Our proposed approach focuses on identifying the interaction of camera wearer with important people. We have used AlexNet convolutional neural network to filter the key-frames (frames where camera wearer interacts closely with the people). We used five convolutional layers and two completely connected hidden layers and an output layer. Dropout regularization method is used to reduce the overfitting problem in completely connected layers. Performance of the proposed method is evaluated on UT Ego standard dataset. Experimental results signify the effectiveness of the proposed method in terms of summarizing the egocentric videos.

show abstract

“…Analysis of images captured by egocentric cameras can reveal a lot about the person recording such images, including, intentions, personality, interests, etc. Firstperson gaze prediction is useful in a wide range of applications in health care, education and entertainment, for tasks such as action and event recognition [35], recognition of handled objects [37], discovering important people [16], video re-editing [26], video summarization [45], engagement detection [42], and assistive vision systems [18].…”

Section: Introductionmentioning

confidence: 99%

Digging Deeper Into Egocentric Gaze Prediction

Tavakoli

Rahtu

Kannala

et al. 2019

2019 IEEE Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

This paper digs deeper into factors that influence egocentric gaze. Instead of training deep models for this purpose in a blind manner, we propose to inspect factors that contribute to gaze guidance during daily tasks. Bottom-up saliency and optical flow are assessed versus strong spatial prior baselines. Task-specific cues such as vanishing point, manipulation point, and hand regions are analyzed as representatives of top-down information. We also look into the contribution of these factors by investigating a simple recurrent neural model for ego-centric gaze prediction. First, deep features are extracted for all input video frames. Then, a gated recurrent unit is employed to integrate information over time and to predict the next fixation. We also propose an integrated model that combines the recurrent model with several top-down and bottom-up cues. Extensive experiments over multiple datasets reveal that (1) spatial biases are strong in egocentric videos, (2) bottom-up saliency models perform poorly in predicting gaze and underperform spatial biases, (3) deep features perform better compared to traditional features, (4) as opposed to hand regions, the manipulation point is a strong influential cue for gaze prediction, (5) combining the proposed recurrent model with bottom-up cues, vanishing points and, in particular, manipulation point results in the best gaze prediction accuracy over egocentric videos, (6) the knowledge transfer works best for cases where the tasks or sequences are similar, and (7) task and activity recognition can benefit from gaze prediction. Our findings suggest that (1) there should be more emphasis on hand-object interaction and (2) the egocentric vision community should consider larger datasets including diverse stimuli and more subjects.

show abstract

Detecting Engagement in Egocentric Video

Cited by 45 publications

References 38 publications

Rethinking the Test Collection Methodology for Personal Self-tracking Data

Rethinking the Test Collection Methodology for Personal Self-tracking Data

Egocentric Video Summarization Based on People Interaction Using Deep Learning

Digging Deeper Into Egocentric Gaze Prediction

Contact Info

Product

Resources

About