Immersive Gastronomic Experience with Distributed Reality

Pérez, Pablo; Gonzalez-Sosa, Ester; Kachach, Redouane; Ruiz, Jaime; Benito, Ignacio; Pereira, Francisco; Villegas, Alvaro

doi:10.1109/wevr.2019.8809591

Cited by 22 publications

(9 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this regard, the simplest methods are those based on color-based approaches [2,1,7,6], which can be deployed in real time but tend to work well just under controlled conditions. Besides, the segmented objects are constrained by color appearance.…”

Section: Related Workmentioning

confidence: 99%

Real Time Egocentric Object Segmentation: THU-READ Labeling and Benchmarking Results

Gonzalez-Sosa,

Robledo,

Gonzalez-Morin

et al. 2021

Preprint

View full text Add to dashboard Cite

Egocentric segmentation has attracted recent interest in the computer vision community due to their potential in Mixed Reality (MR) applications. While most previous works have been focused on segmenting egocentric human body parts (mainly hands), little attention has been given to egocentric objects. Due to the lack of datasets of pixelwise annotations of egocentric objects, in this paper we contribute with a semantic-wise labeling of a subset of 2124 images from the RGB-D THU-READ Dataset. We also report benchmarking results using Thundernet, a real-time semantic segmentation network, that could allow future integration with end-to-end MR applications.

show abstract

Section: Related Workmentioning

confidence: 99%

Real Time Egocentric Object Segmentation: THU-READ Labeling and Benchmarking Results

Gonzalez-Sosa,

Robledo,

Gonzalez-Morin

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In the same line, Perez et al [49] used a YCbCr skin detection algorithm based on red chrominance, adding a transparency alpha layer to the local reality. By using less strict thresholds than those normally used for skin detection, objects with yellow and red tones with high saturation such as food were also segmented.…”

Section: A Color-based Approachesmentioning

confidence: 99%

Enhanced Self-Perception in Mixed Reality: Egocentric Arm Segmentation and Database With Automatic Labeling

et al. 2020

Self Cite

View full text Add to dashboard Cite

In this study, we focus on the egocentric segmentation of arms to improve self-perception in Augmented Virtuality (AV). The main contributions of this work are: i) a comprehensive survey of segmentation algorithms for AV; ii) an Egocentric Arm Segmentation Dataset (EgoArm), composed of more than 10, 000 images, demographically inclusive (variations of skin color, and gender), and open for research purposes. We also provide all details required for the automated generation of groundtruth and semi-synthetic images; iii) the proposal of a deep learning network to segment arms in AV; iv) a detailed quantitative and qualitative evaluation to showcase the usefulness of the deep network and EgoArm dataset, reporting results on different real egocentric hand datasets, including GTEA Gaze+, EDSH, EgoHands, Ego Youtube Hands, THU-Read, TEgO, FPAB, and Ego Gesture, which allow for direct comparisons with existing approaches using color or depth. Results confirm the suitability of the EgoArm dataset for this task, achieving improvements up to 40% with respect to the baseline network, depending on the particular dataset. Results also suggest that, while approaches based on color or depth can work under controlled conditions (lack of occlusion, uniform lighting, only objects of interest in the near range, controlled background, etc.), deep learning is more robust in real AV applications.

show abstract

“…Likewise, Perez et al [46] used a YCbCr skin detection algorithm based on red chrominance, adding a transparency alpha layer to the local reality. By using less strict thresholds than those normally used for skin detection, objects with yellow and red tones with high saturation such as food were also segmented.…”

Section: Color-basedmentioning

confidence: 99%

Enhanced Self-Perception in Mixed Reality: Egocentric Arm Segmentation and Database with Automatic Labelling

Gonzalez-Sosa,

Perez,

Tolosana

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

In this study, we focus on the egocentric segmentation of arms to improve self-perception in Augmented Virtuality (AV). The main contributions of this work are: i) a comprehensive survey of segmentation algorithms for AV; ii) an Egocentric Arm Segmentation Dataset, composed of more than 10, 000 images, comprising variations of skin color, and gender, among others. We provide all details required for the automated generation of groundtruth and semi-synthetic images; iii) the use of deep learning for the first time for segmenting arms in AV; iv) to showcase the usefulness of this database, we report results on different real egocentric hand datasets, including GTEA Gaze+, EDSH, EgoHands, Ego Youtube Hands, THU-Read, TEgO, FPAB, and Ego Gesture, which allow for direct comparisons with existing approaches utilizing color or depth. Results confirm the suitability of the EgoArm dataset for this task, achieving improvement up to 40% with respect to the original network, depending on the particular dataset. Results also suggest that, while approaches based on color or depth can work in controlled conditions (lack of occlusion, uniform lighting, only objects of interest in the near range, controlled background, etc.), egocentric segmentation based on deep learning is more robust in real AV applications.

show abstract

Immersive Gastronomic Experience with Distributed Reality

Cited by 22 publications

References 18 publications

Real Time Egocentric Object Segmentation: THU-READ Labeling and Benchmarking Results

Real Time Egocentric Object Segmentation: THU-READ Labeling and Benchmarking Results

Enhanced Self-Perception in Mixed Reality: Egocentric Arm Segmentation and Database With Automatic Labeling

Enhanced Self-Perception in Mixed Reality: Egocentric Arm Segmentation and Database with Automatic Labelling

Contact Info

Product

Resources

About