Modeling Gliding-based Target Selection for Blind Touchscreen Users

Abstract

Gliding a finger on touchscreen to reach a target, that is, touch exploration, is a common selection method of blind screen-reader users. This paper investigates their gliding behavior and presents a model for their motor performance. We discovered that the gliding trajectories of blind people are a mixture of two strategies: 1) ballistic movements with iterative corrections relying on non-visual feedback, and 2) multiple sub-movements separated by stops, and concatenated until the target is reached. Based on this finding, we propose the mixture pointing model, a model that relates movement time to distance and width of the target. The model outperforms extant models, improving R 2 from 0.65 for Fitts' law to 0.76, and is superior in cross-validation and information criteria. The model advances understanding of gliding-based target selection and serves as a tool for designing interface layouts for screen-reader based touch exploration. CCS CONCEPTS• Human-centered computing → Pointing; Accessibility theory, concepts and paradigms.

Modeling Touch-based Menu Selection Performance of Blind Users via Reinforcement Learning

“…We used Fitts' law to model each fnger traveling action. • Gliding model based method: We used the mixture gliding model for blind users [41] to predict the menu selection time.…”

“…Although these assumptions refected only the expert users' performance, they were necessary for applying the input action models. The parameters of the Fitts' law, gliding model [41], swiping model were the same with the direct touch action model, the gliding action model, and the swiping action model respectively in Table 2 for the one-group model.…”

“…There were some existing works that tried to model the target selection time of blind users based on Fitts' Law [16,17]. Ko et al [41] presented a mixture model for gliding strategies blind users employ when interacting with touchscreens. However, the more generic problem of how blind users make decisions on menu selection strategies is still an open research problem.…”

“…A user moves the fnger from one to another location on the screen while keeping the fnger in contact with the screen in the entire course of gliding. We adopted the gliding-based pointing model for blind users proposed by Ko et al [41] to estimate the mean of time cost for a gliding action. The gliding-based pointing model (Equation 4) [41] states that the mean selection time ( ) for gliding is linearly related to both fnger travel distance and index of difculty of the task = log 2 (/ + 1) where is the width of a target: = + + log 2 ( + 1).…”

“…We adopted the gliding-based pointing model for blind users proposed by Ko et al [41] to estimate the mean of time cost for a gliding action. The gliding-based pointing model (Equation 4) [41] states that the mean selection time ( ) for gliding is linearly related to both fnger travel distance and index of difculty of the task = log 2 (/ + 1) where is the width of a target: = + + log 2 ( + 1). (4) We empirically determine the parameters , , and from gliding data collected in the user study.…”