Gesture Recognition using Wearable Vision Sensors to Enhance Visitors' Museum Experiences

Baraldi, Lorenzo; Paci, F.; Serra, Giuseppe; Benini, Luca; Cucchiara, Rita

doi:10.1109/jsen.2015.2411994

Cited by 31 publications

(33 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…According to survey of [11], the most commonly explored objective of egocentric vision is object recognition and tracking. Furthermore, hands are among the most common objects in the user's field of view, and a proper detection, localization, and tracking could be a main input for other objectives, such as gesture recognition, understanding hand-object interactions, and activity recognition [5,[12][13][14][15][16][17][18][19][20]. Recently, egocentric pixel-level hand detection has attracted more and more attention.…”

Section: Related Workmentioning

confidence: 99%

“…Zhu et al [2] extend the pixel-level method by introducing shape information of pixels based on structured forests. Baraldi et al [5] utilize temporal and spatial coherence strategy to improve the hand segmentation of the pixel-level method. The state of the arts use video clip EDSH1 as the training data and test their approaches on the rest clips of EDSH2 and EDSHK.…”

Section: Evaluation On Benchmark Datasetmentioning

confidence: 99%

See 1 more Smart Citation

Coarse-to-fine online learning for hand segmentation in egocentric video

Zhao

Luo

Quan

2018

J Image Video Proc.

View full text Add to dashboard Cite

Hand segmentation is one of the most fundamental and crucial steps for egocentric human-computer interaction. The special egocentric view brings new challenges to hand segmentation tasks, such as the unpredictable environmental conditions. The performance of traditional hand segmentation methods depend on abundant manually labeled training data. However, these approaches do not appropriately capture the whole properties of egocentric human-computer interaction for neglecting the user-specific context. It is only necessary to build a personalized hand model of the active user. Based on this observation, we propose an online-learning hand segmentation approach without using manually labeled data for training. Our approach consists of top-down classifications and bottom-up optimizations. More specifically, we divide the segmentation task into three parts, a frame-level hand detection which detects the presence of the interactive hand using motion saliency and initializes hand masks for online learning, a superpixel-level hand classification which coarsely segments hand regions from which stable samples are selected for next level, and a pixel-level hand classification which produces a fine-grained hand segmentation. Based on the pixel-level classification result, we update the hand appearance model and optimize the upper layer classifier and detector. This online-learning strategy makes our approach robust to varying illumination conditions and hand appearances. Experimental results demonstrate the robustness of our approach.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Evaluation On Benchmark Datasetmentioning

confidence: 99%

Coarse-to-fine online learning for hand segmentation in egocentric video

Zhao

Luo

Quan

2018

J Image Video Proc.

View full text Add to dashboard Cite

show abstract

“…Specifically, teleoperation, telemanipulation and telepresence have been benefited from these wearable devices that, coupled to robots composed of wheels, a stand and a camera, provide humans with an enhanced control of a robot located in a remote environment [20], [21]. The improvement of immersion and telepresence experience have also been possible by the rapid progress achieved in robot and sensor technology [22], [23], [24]. Particularly, humanoid robots, which try to mimic the human body structure, movements and sensory capabilities, offer a more natural platform for remote control, exploration and interaction with humans and the surrounding environment [25], [26], [27].…”

Section: Related Workmentioning

confidence: 99%

Multisensory Wearable Interface for Immersion and Telepresence in Robotics

2017

View full text Add to dashboard Cite

ReuseUnless indicated otherwise, fulltext items are protected by copyright with all rights reserved. The copyright exception in section 29 of the Copyright, Designs and Patents Act 1988 allows the making of a single copy solely for the purpose of non-commercial research or private study within the limits of fair dealing. The publisher or other rights-holder may allow further reproduction and re-use of this version -refer to the White Rose Research Online record for this item. Where records identify the publisher as the copyright holder, users can verify any specific terms of use on the publisher's website. TakedownIf you consider content in White Rose Research Online to be in breach of UK law, please notify us by emailing eprints@whiterose.ac.uk including the URL of the record and the reason for the withdrawal request.1530-437X (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Abstract-The idea of being present in a remote location has inspired researchers to develop robotic devices that make humans to experience the feeling of telepresence. These devices need of multiple sensory feedback to provide a more realistic telepresence experience. In this work, we develop a wearable interface for immersion and telepresence that provides to human with the capability of both to receive multisensory feedback from vision, touch and audio and to remotely control a robot platform. Multimodal feedback from a remote environment is based on the integration of sensor technologies coupled to the sensory system of the robot platform. Remote control of the robot is achieved by a modularised architecture, which allows to visually explore the remote environment. We validated our work with multiple experiments where participants, located at different venues, were able to successfully control the robot platform while visually exploring, touching and listening a remote environment. In our experiments we used two different robotic platforms: the iCub humanoid robot and the Pioneer LX mobile robot. These experiments show that our wearable interface is comfortable, easy to use and adaptable to different robotic platforms. Furthermore, we observed that our approach allows humans to experience a vivid feeling of being present in a remote environment.

show abstract

“…Wearable optical see-through devices have been researched by Dalens et al (2014) and Baraldi et al (2015) who implemented computer vision methods on devices such as Google Glass to recognize paintings in real time, and detect hand gestures which visitors could use to naturally interact with the artwork. These studies however did not test user experience.…”

Section: Related Workmentioning

confidence: 99%

Comparison of Wearable Optical See-through and Handheld Devices as Platform for an Augmented Reality Museum Guide

Serubugo

Skantarova

Nielsen

et al. 2017

Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Application

View full text Add to dashboard Cite

Abstract:Self-service guides are a common way of providing information about artworks exhibited in museums. Modern advances in handheld mobile applications and wearable optical see-through devices that use augmented reality offer new ways of designing museum guides that are more engaging and interactive than traditional self-service guides such as written descriptions or audio guides. In this study we compare wearable (smart glasses) and handheld (smartphone) devices as a platform for an augmented reality museum guide. We have developed a museum guide for both a smartphone and smart glasses that can identify artwork, direct the visitors' attention to specific details in it, as well as engage them through a game. The platforms are compared based on participants' preference feedback and game performance, and are also evaluated by a coordinator from a collaborating museum. We conclude with a discussion of potentials of these platforms as augmented reality museum guides and suggest promising future work.

show abstract

Gesture Recognition using Wearable Vision Sensors to Enhance Visitors' Museum Experiences

Cited by 31 publications

References 32 publications

Coarse-to-fine online learning for hand segmentation in egocentric video

Coarse-to-fine online learning for hand segmentation in egocentric video

Multisensory Wearable Interface for Immersion and Telepresence in Robotics

Comparison of Wearable Optical See-through and Handheld Devices as Platform for an Augmented Reality Museum Guide

Contact Info

Product

Resources

About