Social Sensemaking with AI: Designing an Open-ended AI Experience with a Blind Child

The 23rd International ACM SIGACCESS Conference on Computers and Accessibility

Sato

Asakawa

et al. 2021

Fig. 1. A pedestrian detection scenario demonstrating the data collection in our study. A blind person, wearing smart glasses with our working prototype, and a sighted person walk toward each other in a corridor. The smart glasses detect the sighted passerby, estimate his proximity, and share his relative location and head pose.The spatial behavior of passersby can be critical to blind individuals to initiate interactions, preserve personal space, or practice social distancing during a pandemic. Among other use cases, wearable cameras employing computer vision can be used to extract proxemic signals of others and thus increase access to the spatial behavior of passersby for blind people. Analyzing data collected in a study with blind (N=10) and sighted (N=40) participants, we explore: (i) visual information on approaching passersby captured by a head-worn camera; (ii) pedestrian detection algorithms for extracting proxemic signals such as passerby presence, relative position, distance, and head pose; and (iii) opportunities and limitations of using wearable cameras for helping blind people access proxemics related to nearby people. Our observations and findings provide insights into dyadic behaviors for assistive pedestrian detection and lead to implications for the design of future head-worn cameras and interactions. CCS Concepts: • Human-centered computing → User studies; Empirical studies in HCI; Mobile devices; Empirical studies in accessibility; Accessibility technologies; • Computing methodologies → Computer vision.

Section: Accessing the Spatial Behavior Of Others With A Cameramentioning

confidence: 99%

Accessing Passersby Proxemic Signals through a Head-Worn Camera: Opportunities and Limitations for the Blind

The 23rd International ACM SIGACCESS Conference on Computers and Accessibility

Sato

Asakawa

et al. 2021

“…Furthermore, these systems do not address the issue of locating and acquiring objects in arm reachable distance, known as the last meter problem [48]. Recent work by Morrison et al [52] prototyped a computer vision based system that processes the real-time image feed for orienting the system, which is similar to our system. However, their system was designed to assist with a blind childâĂŹs social interaction (e.g., conversation engagement).…”

Section: Related Work 21 Object Finding Approachesmentioning

confidence: 99%

AIGuide: Augmented Reality Hand Guidance in a Visual Prosthetic

Aldas

ACM Trans. Access. Comput.

et al. 2022

Locating and grasping objects is a critical task in people’s daily lives. For people with visual impairments, this task can be a daily struggle. The support of augmented reality frameworks in smartphones can overcome the limitations of current object detection applications designed for people with visual impairments. We present AIGuide, a self-contained smartphone application that leverages augmented reality technology to help users locate and pick up objects around them. We conducted a user study to investigate the effectiveness of AIGuide in a visual prosthetic for providing guidance; compare it to other assistive technology form factors; investigate the use of multimodal feedback, and provide feedback about the overall experience. We gathered performance data and participants’ reactions and analyzed videos to understand users’ interactions with the nonvisual smartphone user interface. Our results show that AIGuide is a promising technology to help people with visual impairments locate and acquire objects in their daily routine. The benefits of AIGuide may be enhanced with appropriate interaction design.

“…The use of smart glasses by people with visual impairments is not new in the accessibility field. Typically, prior work has been interested in automatically leveraging the input from smart glasses camera to support people with visual impairments navigate indoors [5,18,54], read text [22,47,55], or detect objects [16] and people [28,36,48] without the explicit help from sighted people. Assuming that the field of view of the camera worn by users with visual impairments can capture the area of interest to them, all the prior work proposes assistive systems that interpret the visual input and convert it into non-visual formats, such as audio and haptic.…”

Section: People With Visual Impairments Remote Guidance and Asssistiv...mentioning

confidence: 99%

From the Lab to People's Home: Lessons from Accessing Blind Participants' Interactions via Smart Glasses in Remote Studies

Lee,

Hong,

Jarjue

et al. 2022

Preprint

Researchers have adopted remote methods, such as online surveys and video conferencing, to overcome challenges in conducting inperson usability testing, such as participation, user representation, and safety. However, remote user evaluation on hardware testbeds is limited, especially for blind participants, as such methods restrict access to observations of user interactions. We employ smart glasses in usability testing with blind people and share our lessons from a case study conducted in blind participants' homes (𝑁 =12), where the experimenter can access participants' activities via dual video conferencing: a third-person view via a laptop camera and a first-person view via smart glasses worn by the participant. We show that smart glasses hold potential for observing participants' interactions with smartphone testbeds remotely; on average 58.7% of the interactions were fully captured via the first-person view compared to 3.7% via the third-person. However, this gain is not uniform across participants as it is susceptible to head movements orienting the ear towards a sound source, which highlights the need for a more inclusive camera form factor. We also share our lessons learned when it comes to dealing with lack of screen reader support in smart glasses, a rapidly draining battery, and Internet connectivity in remote studies with blind participants. CCS CONCEPTS• Human-centered computing → Accessibility design and evaluation methods.