Making touchscreen keyboards adaptive to keys, hand postures, and individuals

Abstract

We propose a new approach for improving text entry accuracy on touchscreen keyboards by adapting the underlying spatial model to factors such as input hand postures, individuals, and target key positions. To combine these factors together, we introduce a hierarchical spatial backoff model (SBM) that consists of submodels with different levels of complexity. The most general model includes no adaptive factors, whereas the most specific model includes all three. Considering that in practice people may switch hand postures (e.g., from twothumb to one-finger) to better suit a situation, and that the specific submodels may take time to train for each user, a specific submodel should be applied only if its corresponding input posture can be identified with confidence, and if the submodel has enough training data from the user. We introduce the backoff mechanism to fall back to a simpler model if either of these conditions are not met. We implemented a prototype system capable of reducing the language-modelindependent error rate by 13.2% using an online posture classifier with 86.4% accuracy. Further improvements in error rate may be possible with even better posture classification.

Nomadic Input on Mobile Devices: The Influence of Touch Input Technique and Walking Speed on Performance and Offset Modeling

“…A comparison with adjusted p values (Holm-Bonferroni method) over all subjects and methods for different speeds and over all subjects and speeds for different input techniques was performed (due to the large number of tests and 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 It is interesting to observe that for all measurement conditions the resulting adjusted pvalues suggest there is a different underlying process responsible for the generation of the observed densities. These results seem to suggest that for a particular walking speed, the gait phase data could be used as an additional feature (especially in on-the-go scenario) to infer which input technique is being used and possibly improve on detection rates reported in the literature (Goel, Wobbrock & Patel, 2012, Ouyang, Partridge & Zhai, 2013. The same analysis could be achieved for Figure 17b, i.e.…”

“…The authors also noted that the typing speed (expressed in words per minute) was not significantly affected with inclusion of a posture-adaptive keyboard. This approach was further refined in (Yin, Ouyang, Partridge & Zhai, 2013) to include a user-specific posture adaptive keyboard. In the paper, an elaborate hierarchical model was used which reduced the language-model independent error rate by 13.2% over the baseline model.…”

Investigating the effects of encumbrance on one- and two- handed interactions with mobile devices

“…Later, Goel et al [7] examined the user's touch pattern to distinguish between the three input postures to improve text entry. Yin et al [27] also used hand posture to enhance text entry on touchscreen keyboards and reported that their method could differentiate single finger and two thumbs typing with an accuracy of 86.4%.…”

Observations on Typing from 136 Million Keystrokes

“…Several papers have examined typing dynamics and gesturing on small physical keyboards and mobile (soft) keyboards (e.g. [3,9,13,25,35,43]). The papers considering modern typing have involved laboratory studies with small sample sizes.…”