WiseType: A Tablet Keyboard with Color-Coded Visualization and Various Editing Options for Error Correction

Alharbi, Ohoud; Arif, Ahmed Sabbir; Stuerzlinger, Wolfgang; Dunlop, Mark; Komninos, Andreas

doi:10.20380/gi2019.04

Cited by 9 publications

(5 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Compound errors in uni-modal text entry systems cause a generally linear increase in correction time [3]. Recent work has shown that improved error correction methods can reduce the time it takes users to reconcile text entry errors, decreasing the overall amount the user is slowed down by the error correction process [1]. Further work is needed to examine whether this holds true for multimodal interactions.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Understanding Gesture and Speech Multimodal Interactions for Manipulation Tasks in Augmented Reality Using Unconstrained Elicitation

Williams¹,

Ortega²

2020

Preprint

View full text Add to dashboard Cite

This research establishes a better understanding of the syntax choices in speech interactions and of how speech, gesture, and multimodal gesture and speech interactions are produced by users in unconstrained object manipulation environments using augmented reality.The work presents a multimodal elicitation study conducted with 24 participants. The canonical referents for translation, rotation, and scale were used along with some abstract referents (create, destroy, and select). In this study time windows for gesture and speech multimodal interactions are developed using the start and stop times of gestures and speech as well as the stoke times for gestures.While gestures commonly precede speech by 81 ms we find that the stroke of the gesture is commonly within 10 ms of the start of speech. Indicating that the information content of a gesture and its co-occurring speech are well aligned to each other. Lastly, the trends across the most common proposals for each modality are examined. Showing that the disagreement between proposals is often caused by a variation of hand posture or syntax. Allowing us to present aliasing recommendations to increase the percentage of users' natural interactions captured by future multimodal interactive systems.CCS Concepts: • Human-centered computing → Human computer interaction (HCI); Interaction techniques; Gestural input; Empirical studies in interaction design; User centered design.

show abstract

Section: Discussionmentioning

confidence: 99%

“…The metrics used for gesture proposal interpretation are Agreement Rate (AR ) 1 , co-agreement rate (CAR ) , and the (V r d ) significance test [57,58,61]. AR is the proportion of proposals in agreement over the total possible proposals pairs in agreement.…”

Section: Methodsmentioning

confidence: 99%

Understanding Gesture and Speech Multimodal Interactions for Manipulation Tasks in Augmented Reality Using Unconstrained Elicitation

Williams¹,

Ortega²

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…It helps to reduce the operations on deleting and positioning operations in word correction. WiseType [ 1 ] introduced novel visualization to highlight errors to assist error correction. Besides, grammar checking methods on entered text such as Gboard [ 33 ] and Grammarly [ 23 ] support correcting words and revising text by providing possible text on the suggestion bar.…”

Section: Related Workmentioning

confidence: 99%

“…However, these approaches do not support the multimodal interaction to combine the voice input with touch gesture operations. VT also has a larger scope than previous research systems such as WiseType [ 1 ], Gestures and Widgets [ 15 ], TouchTap [ 16 ], Gaze’N’Touch [ 44 ] and Gedit [ 55 ]. VT focuses on both text formatting and error correction, while those systems focus on only one aspect of text editing: Gestures and Widgets [ 15 ], TouchTap [ 16 ] and Gedit [ 55 ] were about text formatting only; Gaze’N’Touch [ 44 ] was about improving text selection; WiseType [ 1 ] introduced novel visualization to highlight errors to assist error correction.…”

Section: Related Workmentioning

confidence: 99%

“…VT also has a larger scope than previous research systems such as WiseType [ 1 ], Gestures and Widgets [ 15 ], TouchTap [ 16 ], Gaze’N’Touch [ 44 ] and Gedit [ 55 ]. VT focuses on both text formatting and error correction, while those systems focus on only one aspect of text editing: Gestures and Widgets [ 15 ], TouchTap [ 16 ] and Gedit [ 55 ] were about text formatting only; Gaze’N’Touch [ 44 ] was about improving text selection; WiseType [ 1 ] introduced novel visualization to highlight errors to assist error correction. Additionally WiseType [ 1 ], Gestures and Widgets [ 15 ], TouchTap [ 16 ], and Gedit [ 55 ] involve only touch input, while VT integrates two modalities (touch and voice).…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Voice and Touch Based Error-tolerant Multimodal Text Editing and Correction for Smartphones

Zhao

Cui

Ramakrishnan

et al. 2021

The 34th Annual ACM Symposium on User Interface Software and Technology

View full text Add to dashboard Cite

Editing operations such as cut, copy, paste, and correcting errors in typed text are often tedious and challenging to perform on smartphones. In this paper, we present VT, a voice and touch-based multi-modal text editing and correction method for smartphones. To edit text with VT, the user glides over a text fragment with a finger and dictates a command, such as “bold” to change the format of the fragment, or the user can tap inside a text area and speak a command such as “highlight this paragraph” to edit the text. For text correcting, the user taps approximately at the area of erroneous text fragment and dictates the new content for substitution or insertion. VT combines touch and voice inputs with language context such as language model and phrase similarity to infer a user’s editing intention, which can handle ambiguities and noisy input signals. It is a great advantage over the existing error correction methods (e.g., iOS’s Voice Control) which require precise cursor control or text selection. Our evaluation shows that VT significantly improves the efficiency of text editing and text correcting on smartphones over the touch-only method and the iOS’s Voice Control method. Our user studies showed that VT reduced the text editing time by 30.80%, and text correcting time by 29.97% over the touch-only method. VT reduced the text editing time by 30.81%, and text correcting time by 47.96% over the iOS’s Voice Control method.

show abstract

LEDBoard: Using Visual Feedback to Support Text Entry with Physical Keyboards

Komninos

Kavvathas

Simou

2022

Proceedings of the International Conference on Ubiquitous Computing &Amp; Ambient Intelligence (UCAmI 2022)

View full text Add to dashboard Cite

WiseType: A Tablet Keyboard with Color-Coded Visualization and Various Editing Options for Error Correction

Cited by 9 publications

References 37 publications

Understanding Gesture and Speech Multimodal Interactions for Manipulation Tasks in Augmented Reality Using Unconstrained Elicitation

Understanding Gesture and Speech Multimodal Interactions for Manipulation Tasks in Augmented Reality Using Unconstrained Elicitation

Voice and Touch Based Error-tolerant Multimodal Text Editing and Correction for Smartphones

LEDBoard: Using Visual Feedback to Support Text Entry with Physical Keyboards

Contact Info

Product

Resources

About