Multimodal Egocentric Analysis of Focused Interactions

Bano, Sophia; Suveges, Tamas; Zhang, Jianguo; McKenna, Stephen J.

doi:10.1109/access.2018.2850284

Cited by 17 publications

(14 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In (Bolaños and Radeva, 2016) a two-step method is presented for food detection and recognition in lifelogging images. A recognition of personal locations in daily activities of the wearer is studied in (Furnari et al, 2017) while social interactions and lifestyle patterns are analyzed in (Bano et al, 2018) and (Herruzo et al, 2017) respectively. Visual lifelogging has also been used in ambient assisted living applications (Climent-Pérez et al, 2020) such as fall detection, monitoring, etc.…”

Section: Background 21 Visual Lifeloggingmentioning

confidence: 99%

“…The dataset was used for training systems to recognize hand actions. The abundance and availability of wearable cameras and smartphones have led to the creation of several first person datasets the last years (Bolaños et al, 2015) including datasets for object recognition (Bullock et al, 2015), activity recognition (Gurrin et al, 2016), social interaction analysis (Bano et al, 2018), etc. However, not all datasets are publicly available to the academic community.…”

Section: Egocentric Databasesmentioning

confidence: 99%

See 1 more Smart Citation

A First-person Database for Detecting Barriers for Pedestrians

Θεοδοσίου

Partaourides

Atun

et al. 2020

Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Application

View full text Add to dashboard Cite

Egocentric vision, which relates to the continuous interpretation of images captured by wearable cameras, is increasingly being utilized in several applications to enhance the quality of citizens' life, especially for those with visual or motion impairments. The development of sophisticated egocentric computer vision techniques requires automatic analysis of large databases of first-person point of view visual data collected through wearable devices. In this paper, we present our initial findings regarding the use of wearable cameras for enhancing the pedestrians' safety while walking in city sidewalks. For this purpose, we create a first-person database that entails annotations on common barriers that may put pedestrians in danger. Furthermore, we derive a framework for collecting visual lifelogging data and define 24 different categories of sidewalk barriers. Our dataset consists of 1796 annotated images covering 1969 instances of barriers. The analysis of the dataset by means of object classification algorithms, depict encouraging results for further study.

show abstract

Section: Background 21 Visual Lifeloggingmentioning

confidence: 99%

Section: Egocentric Databasesmentioning

confidence: 99%

A First-person Database for Detecting Barriers for Pedestrians

Θεοδοσίου

Partaourides

Atun

et al. 2020

Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Application

View full text Add to dashboard Cite

show abstract

“…Recently, wearable cameras have enabled the automatic capture of social life in a naturalistic setting, from a first-person point of view [1]. This has opened the unique opportunity of analyzing the real involvement in social interactions at the personal level [2][3][4]. However, the research focus in the egocentric vision domain has been so far on the detection [3,4] and classification of social interactions based on the kind of relations [5,6], while the problem of fine-grained classification of a specific relation based on the degree of interactivity has been addressed only in [2].…”

Section: Introductionmentioning

confidence: 99%

“…This has opened the unique opportunity of analyzing the real involvement in social interactions at the personal level [2][3][4]. However, the research focus in the egocentric vision domain has been so far on the detection [3,4] and classification of social interactions based on the kind of relations [5,6], while the problem of fine-grained classification of a specific relation based on the degree of interactivity has been addressed only in [2]. This latter categorization would be of paramount importance to truly understand social interactions and thus to allow an easier human-machine communication.…”

Section: Introductionmentioning

confidence: 99%

Interaction-GCN: A Graph Convolutional Network Based Framework for Social Interaction Recognition in Egocentric Videos

Felicioni

Dimiccoli

2021

2021 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

In this paper we propose a new framework to categorize social interactions in egocentric videos, we named Interac-tionGCN. Our method extracts patterns of relational and nonrelational cues at the frame level and uses them to build a relational graph from which the interactional context at the frame level is estimated via a Graph Convolutional Network (GCN) based approach. Then it propagates this context over time, together with first-person motion information, through a Gated Recurrent Unit architecture. Ablation studies and experimental evaluation on two publicly available datasets validate the proposed approach and establish state of the art results. 1

show abstract

“…The objective of the proposed research work is to exploit these properties for summarization of egocentric videos. Egocentric video summarization has useful applications in many domains, i.e., law enforcement [1,3], health care [4], surveillance [5], sports [6], and media [7,8].…”

Section: Introductionmentioning

confidence: 99%

Egocentric Video Summarization Based on People Interaction Using Deep Learning

Ghafoor

Javed

Irtaza

et al. 2018

Mathematical Problems in Engineering

View full text Add to dashboard Cite

The availability of wearable cameras in the consumer market has motivated the users to record their daily life activities and post them on the social media. This exponential growth of egocentric videos demand to develop automated techniques to effectively summarizes the first-person video data. Egocentric videos are commonly used to record lifelogs these days due to the availability of low cost wearable cameras. However, egocentric videos are challenging to process due to the fact that placement of camera results in a video which presents great deal of variation in object appearance, illumination conditions, and movement. This paper presents an egocentric video summarization framework based on detecting important people in the video. The proposed method generates a compact summary of egocentric videos that contains information of the people whom the camera wearer interacts with. Our proposed approach focuses on identifying the interaction of camera wearer with important people. We have used AlexNet convolutional neural network to filter the key-frames (frames where camera wearer interacts closely with the people). We used five convolutional layers and two completely connected hidden layers and an output layer. Dropout regularization method is used to reduce the overfitting problem in completely connected layers. Performance of the proposed method is evaluated on UT Ego standard dataset. Experimental results signify the effectiveness of the proposed method in terms of summarizing the egocentric videos.

show abstract

Multimodal Egocentric Analysis of Focused Interactions

Cited by 17 publications

References 33 publications

A First-person Database for Detecting Barriers for Pedestrians

A First-person Database for Detecting Barriers for Pedestrians

Interaction-GCN: A Graph Convolutional Network Based Framework for Social Interaction Recognition in Egocentric Videos

Egocentric Video Summarization Based on People Interaction Using Deep Learning

Contact Info

Product

Resources

About