Independent mobility is one of the main challenges for blind or visually impaired (BVI) people. In particular, BVI people often need to identify and avoid nearby obstacles, for example a bicycle parked on the sidewalk. This is generally achieved with a combination of residual vision, hearing and haptic sensing using the white cane. However, in many cases, BVI people can only perceive obstacles at short distance (typically about 1m, i.e., the white cane detection range), in other situations obstacles are hard to detect (e.g., those elevated from the ground), while others should not be hit by the white cane (e.g., a standing person). Thus, some time and effort are required to identify the object in order to understand how to avoid it.A solution to these problems can be found in recent computer vision techniques that can run on mobile and wearable devices to detect obstacles at a distance. However, in addition to detecting obstacles, it is also necessary to convey information about them to a BVI user. This contribution presents WatchOut, a sonifcation technique for conveying real-time information about the main characteristics of an obstacle to a BVI person, who can then use this additional feedback to safely navigate in the environment. WatchOut was designed with a user-centric approach, involving two iterations of online questionnaires with BVI participants in order to defne, improve and evaluate the sonifcation technique. WatchOut was implemented and tested as a module of a mobile app that detects obstacles using state-of-the-art computer vision technology. Results show that the system is considered usable, and can guide the users to avoid more than 85% of the obstacles.

We illustrate our experience, gained over years of involvement in multiple research and commercial projects, in developing accessible mobile apps with cross-platform development frameworks (CPDF). These frameworks allow the developers to write the app code only once and run it on both iOS and Android. However, they have limited support for accessibility features, in particular for what concerns the interaction with the system screen reader.To study the coverage of accessibility features in CPDFs, we first systematically analyze screen reader APIs available in native iOS and Android, and we examine whether and at what level the same functionalities are available in two popular CPDF: Xamarin and React Native. This analysis unveils that there are many functionalities shared between native iOS and Android APIs, but most of them are not available neither in React Native nor in Xamarin. In particular, not even all basic APIs are exposed by the examined CPDF. Accessing the unavailable APIs is still possible, but it requires additional effort by the developers who need to write platform-specific code in native APIs, hence partially negating the advantages of CPDF.To address this problem, we consider a representative set of native APIs that cannot be directly accessed from React Native and Xamarin and we report challenges encountered in accessing them. CCS CONCEPTS• Human-centered computing → UI toolkits; Accessibility.

No abstract

People with severe visual impairments usually have no way of identifying the colors of objects in their environment. While existing smartphone apps can recognize colors and speak them aloud, they require the user to center the object of interest in the camera’s field of view, which is challenging for many users. We developed a smartphone app to address this problem that reads aloud the color of the object pointed to by the user’s fingertip, without confusion from background colors. We evaluated the app with nine people who are blind, demonstrating the app’s effectiveness and suggesting directions for improvements in the future.

No abstract