Handwriting for Text Input and the Impact of XR Displays, Surface Alignments, and Sentence Complexities

Abstract

In our study, participants used handwriting for text input in Virtual Reality (VR) (A, E) or Video See-Through Augmented Reality (VST AR) (B, F) on physically aligned (A, B, C) and mid-air (D, E, F) surfaces. For VR, we created a digital twin of the real room and aligned it precisely. The controller was held in a pen-like posture to mimic a stylus. To detect surface contact, we used the controller's pressure-sensitive tip on the physical surface and an orthogonal distance-based approach in mid-air. Participants wrote simple and complex sentences. The example shows the simple sentence, "It should be sunny tomorrow."