JustPoint

Mascetti, Sergio; Gerino, Andrea; Bernareggi, Cristian; D'Acquisto, Silvia; Ducci, Mattia; Coughlan, James M.

doi:10.1145/3132525.3134802

Cited by 3 publications

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Hands Holding Clues for Object Recognition in Teachable Machines

Lee

Kacorri

2019

Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

Camera manipulation confounds the use of object recognition applications by blind people. This is exacerbated when photos from this population are also used to train models, as with teachable machines, where out-of-frame or partially included objects against cluttered backgrounds degrade performance. Leveraging prior evidence on the ability of blind people to coordinate hand movements using proprioception, we propose a deep learning system that jointly models hand segmentation and object localization for object classification. We investigate the utility of hands as a natural interface for including and indicating the object of interest in the camera frame. We confirm the potential of this approach by analyzing existing datasets from people with visual impairments for object recognition. With a new publicly available egocentric dataset and an extensive error analysis, we provide insights into this approach in the context of teachable recognizers.

show abstract

Hands Holding Clues for Object Recognition in Teachable Machines

Lee

Kacorri

2019

Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

show abstract

SonoHaptics: An Audio-Haptic Cursor for Gaze-Based Object Selection in XR

Cho,

Sendhilnathan,

Nebeling

et al. 2024

Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology

View full text Add to dashboard Cite

We introduce SonoHaptics, an audio-haptic cursor for gaze-based 3D object selection. SonoHaptics addresses challenges around providing accurate visual feedback during gaze-based selection in Extended Reality (XR), e. g., lack of world-locked displays in no-or limited-display smart glasses and visual inconsistencies. To enable users to distinguish objects without visual feedback, SonoHaptics employs the concept of cross-modal correspondence in human perception to map visual features of objects (color, size, position,

show abstract

A roadmap towards mobile applications with accessibility for visually impaired users: guideline and its evaluation

Correia¹,

Penha²,

Macedo³

et al. 2019

DeT

View full text Add to dashboard Cite

When applications intend to support accessibility, several aspects of interface design and usability must be reviewed to adapt or extend common functional requirements that are implemented to ensure easy use of applications. However, initiatives to develop guidelines for accessible mobile applications are recent and several approaches present only suggestions rather than a concrete list of requirements. It is becoming increasingly apparent not only the need for social and digital inclusion of people with disabilities but also the urgency to make devices and applications truly accessible to anyone. This work focuses on accessibility requirements for visually impaired users and it is part of a broader effort that involves different types of impairments. Our research method was based on three main steps. First, from an initial set of 1014 scientific and technical articles, 46 were analyzed to identify requirements that are related to visual impairments. Second, we carried out an observation-based analysis to confirm the importance of these requirements, considering a set of accessibility tests with impaired volunteers. Third, the Design Thinking process was used to implement an application prototype, considering the most important requirements that were identified along with the two first steps. This application was submitted to a user-centered evaluation with volunteers, which also supported the evolution of our requirements. We sought to include prior knowledge aligned with all pre-identified requirements, culminating in something that would add to the procedure greater validation. As the main result, this paper presents the complete roadmap of this process, which resulted in a guideline and evaluation method to support the integration of accessibility aspects into mobile applications.

show abstract

JustPoint

Cited by 3 publications

References 3 publications

Hands Holding Clues for Object Recognition in Teachable Machines

Hands Holding Clues for Object Recognition in Teachable Machines

SonoHaptics: An Audio-Haptic Cursor for Gaze-Based Object Selection in XR

A roadmap towards mobile applications with accessibility for visually impaired users: guideline and its evaluation

Contact Info

Product

Resources

About