An assistive EyeWear prototype that interactively converts 3D object locations into spatial audio

Tang, Titus

doi:10.1145/2641248.2642732

Cited by 4 publications

(3 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Due to a limited number of truly blind participants and challenges to find them, we involved blindfolded participants, who simulated the disability being studied. This approach has been widely used in HCI studies [42]- [44]. Here, we recruited sighted, blind and blindfolded participants for the user experiment.…”

Section: Methodsmentioning

confidence: 99%

An Evaluation of a Wearable Assistive Device for Augmenting Social Interactions

Qiu

Han

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Gaze behaviors contain rich information regarding a person's emotions and engagements. Reciprocal eye contact can invoke feelings of liking between two strangers. But blind people cannot perceive and establish the eye contact with sighted counterparts, causing their feelings of social isolation and low confidence in conversations. Thus, our research purpose is to let blind people perceive and react gaze behaviors in social interactions. A Social Glasses system has been implemented iteratively to deliver the multisensory feedback channels of the "eye contact", integrating both visual and tactile feedback. Specifically, the system consists of a Social Glasses device and a tactile wristband, which are worn by a blind person. The Social Glasses simulates the natural gaze for the blind person, aiming at establishing the "eye contact" between blind and sighted people. The tactile wristband enables the blind person to perceive the corresponding tactile feedback when an "eye contact" happens. To test the system, we conducted a user experiment with 40 participants, including 10 blind-sighted pairs (N=20) and 10 blindfolded-sighted pairs (N=20), to see how it could help increase the communication quality between blind and sighted people, as well as to suggest implications for its design. Our main findings demonstrated that both the simulated gaze and the tactile feedback were significantly effective to enhance the communication quality in blind-sighted conversations. Overall, we contribute (1) empirical research findings on how a Social Glasses system enhances the communication quality in blind-sighted conversations; and (2) design principles to inform future assistive wearable device for augmenting social interactions. INDEX TERMS Accessibility, artificial gaze, assistive technology, communication quality, eye tracking, face-to-face communication, gaze contact, visual impairments.

show abstract

Section: Methodsmentioning

confidence: 99%

An Evaluation of a Wearable Assistive Device for Augmenting Social Interactions

Qiu

Han

et al. 2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Much like how we use our eyes, firstperson vision techniques can act as an artificial visual system that perceives the world around camera wearers and assist them to decide on what to do next. Recent work has focused on a variety of assistive technologies such as blind navigation [20,39], object echo-location [38], and person- alized object recognition [15].…”

Section: Introductionmentioning

confidence: 99%

Future Person Localization in First-Person Videos

Yagi

Mangalam

Yonetani

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

173

139

View full text Add to dashboard Cite

We present a new task that predicts future locations of people observed in first-person videos. Consider a firstperson video stream continuously recorded by a wearable camera. Given a short clip of a person that is extracted from the complete stream, we aim to predict that person's location in future frames. To facilitate this future person localization ability, we make the following three key observations: a) First-person videos typically involve significant ego-motion which greatly affects the location of the target person in future frames; b) Scales of the target person act as a salient cue to estimate a perspective effect in first-person videos; c) First-person videos often capture people up-close, making it easier to leverage target poses (e.g., where they look) for predicting their future locations. We incorporate these three observations into a prediction framework with a multi-stream convolution-deconvolution architecture. Experimental results reveal our method to be effective on our new dataset as well as on a public social interaction dataset.

show abstract

“…Around the world hundreds of millions of camera-equipped mobile phones can be used to capture special moments in life. Novel wearable camera technologies (e.g., the Google Glass or the Narrative lapel camera) also offer a new paradigm for keeping a visual record of everyday life in the form of first-person point-of-view videos and can be used to aid productivity, such as automatic activity summarization [1], [2], [3], [4] and assistive systems [5], [6], [7].…”

Section: Introductionmentioning

confidence: 99%

Ego-Surfing: Person Localization in First-Person Videos Using Ego-Motion Signatures

Yonetani

Sato

2018

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Abstract-We envision a future time when wearable cameras are worn by the masses and recording first-person point-of-view videos of everyday life. While these cameras can enable new assistive technologies and novel research challenges, they also raise serious privacy concerns. For example, first-person videos passively recorded by wearable cameras will necessarily include anyone who comes into the view of a camera -with or without consent. Motivated by these benefits and risks, we developed a self-search technique tailored to first-person videos. The key observation of our work is that the egocentric head motion of a target person (i.e., the self) is observed both in the point-of-view video of the target and observer. The motion correlation between the target person's video and the observer's video can then be used to identify instances of the self uniquely. We incorporate this feature into the proposed approach that computes the motion correlation over densely-sampled trajectories to search for a target individual in observer videos. Our approach significantly improves self-search performance over several well-known face detectors and recognizers. Furthermore, we show how our approach can enable several practical applications such as privacy filtering, target video retrieval, and social group clustering.

show abstract

An assistive EyeWear prototype that interactively converts 3D object locations into spatial audio

Cited by 4 publications

References 11 publications

An Evaluation of a Wearable Assistive Device for Augmenting Social Interactions

An Evaluation of a Wearable Assistive Device for Augmenting Social Interactions

Future Person Localization in First-Person Videos

Ego-Surfing: Person Localization in First-Person Videos Using Ego-Motion Signatures

Contact Info

Product

Resources

About