Smart Touch

Abstract

We present two contributions for improving the accessibility of touch screens for people with motor impairments. First, we provide an exploration of the touch behaviors of 10 people with motor impairments, e.g., we describe how touching with the back or sides of the hand, with multiple fingers, or with the knuckles creates varied multi-point touches. Second, we introduce Smart Touch, a novel template-matching technique for touch input that maps any number of arbitrary contact-areas to a user's intended (,) target location. The result is that users with motor impairments can touch however their abilities allow, and Smart Touch will resolve their intended touch point. Smart Touch therefore allows users to touch targets in whichever ways are most comfortable and natural for them. In an experiment, we found that Smart Touch predicted the (,) coordinates of users' intended target locations over three times closer to actual intended targets than the native Landon and Lift-off techniques reported by the built-in touch sensors found in the Microsoft PixelSense interactive tabletop. This result is an important step toward improving touch accuracy for people with motor impairments and others for whom touch screen operation was previously difficult or impossible.

Comparing Touchscreen and Mouse Input Performance by People With and Without Upper Body Motor Impairments

“…Indeed, for users with motor impairments, touchscreen tapping (i.e., pointing) errors were more than three times as high with the touchscreen than with the mouse. Tapping is by far the most common touchscreen interaction technique, which emphasizes the importance of efforts to improve tapping accuracy (e.g., [10,20]).…”

“…While spurious touches are ignored in typical Fitts task (as in our earlier analysis), they can create problems in real systems, for example, causing an intended tap to result in a multi-finger gesture like zoom. While Mott et al [10] report on the number of touchpoints on the screen during tapping tasks, participants in that study tapped in a "comfortable and natural" way, rather than under standard conditions. To our knowledge, spurious touches have not been reported in detail for users with motor impairments.…”

“…Despite the above body of work, controlled studies of touchscreen input performance for people with upper body motor impairments are relatively sparse. Studies have compared novel techniques (e.g., swabbing [17]) to a control condition such as tapping [9,10,17,20], but do not shed light on more general questions related to device comparisons (e.g., touchscreen vs. mouse) and user groups. Other studies have compared tapping input performance for users with and without motor impairments, but using vertical touchscreen kiosks (e.g., [6,14]), where the large target size and device orientation do not directly translate to mobile touchscreens.…”

Toward fairness in AI for people with disabilities SBG@a research roadmap

“…For example, gesture recognition systems 2 are unlikely to work well for people with differences in morphology (e.g., a person with an amputated arm may be unable to perform bimanual gestures, or may grip a device differently than expected; a person with polydactyly's style of touching a screen may register an unanticipated pattern). Failure of gesture recognition systems is also likely in cases where disability affects the nature of motion itself, such as for someone who experiences tremor or spastic motion [56,57]. Fatigue may also impact gesture performance (and therefore recognition accuracy) over time, particularly for groups that may be more susceptible to fatigue such as due to disability or advanced age.…”

Designing and implementing accessible wearable interactions for people with motor impairments

“…For common touchscreen gestures like tapping, crossing and directional gesturing, Guerreiro et al [45] found that targets located at the bottom of the screen and next to the preferred hand were the easiest to select. Specific to tap gestures, while Montague et al [92] investigated touchscreen interaction behaviors in the wild and built and evaluated a novel approach to accommodate individual differences, Mott et al [94] built and evaluated a technique to accommodate multiple touch points to map the user's intended behavior. These studies highlight that mobile devices like smartphones and tablets can offer benefits like independence, but accessibility challenges still exist.…”