Push, Tap, Dwell, and Pinch: Evaluation of Four Mid-air Selection Methods Augmented with Ultrasonic Haptic Feedback

Abstract

This work compares four mid-air target selection methods (Push, Tap, Dwell, Pinch) with two types of ultrasonic haptic feedback (Select, HoverSelect) in a Fitts’ law experiment. Results revealed that Tap is the fastest, the most accurate, and one of the least physically and cognitively demanding selection methods. Pinch is relatively fast but error prone and physically and cognitively demanding. Dwell is slowest by design, yet the most accurate and the least physically and cognitively demanding. Both haptic feedback methods improve selection performance by increasing users’ spatial awareness. Particularly, Push augmented with Hover & Select feedback is comparable to Tap. Besides, participants perceive the selection methods as faster, more accurate, and more physically and cognitively comfortable with the haptic feedback methods.

Mid-Air Gestural Interaction with a Large Fogscreen

“…Furthermore, the results are comparable with the values of 2.10-2.25 bps for mid-air gesture selection with large conventional displays, where selection confirmation was executed by voice or button press [18,19] but lower than the values of 4-6 bps for large touchscreen interaction [34,35]. Regarding dwell-based selection with fog, its throughput is also comparable to the values of 1.7-1.9 bps previously reported [15,21]. These numbers allow us to assume that a tapping gesture performed in the fog is simpler for users to carry out than a tapping gesture performed solely in the air.…”

“…Thus, our results for throughput values for mid-air tapping with fog are 32% higher than the 1.75 bps reported previously for pure mid-air tapping [15]. Furthermore, the results are comparable with the values of 2.10-2.25 bps for mid-air gesture selection with large conventional displays, where selection confirmation was executed by voice or button press [18,19] but lower than the values of 4-6 bps for large touchscreen interaction [34,35].…”

“…A higher throughput may be explained by the fast task completion time of the tapping gesture, while the dwellbased selection gesture had a delay of 1000 ms. The speed superiority of the tapping gesture compared to the dwell-based gesture (500-800 ms) has been reported in previous studies [12,15]. A low number of re-entries for the tapping gesture was not expected because other researchers that compared tapping and dwell-based gestures in interaction with conventional screens have reported opposite findings [12,15].…”

“…Different researchers favored different gestures for selection tasks, making a choice based on the gestures' salient characteristics. For example, dwell-based interaction is intuitive and accurate [12][13][14], whereas tapping gestures are fast and consistent with touchscreen interaction [15][16][17]. Some researchers combined mid-air gestural techniques with other modalities, such as the voice [18] or physical buttons [19].…”

“…Some researchers combined mid-air gestural techniques with other modalities, such as the voice [18] or physical buttons [19]. Furthermore, a user preference for tactile feedback during interactions with touchless user interfaces via gestural inputs has been reported by other studies [15,20]. Some technological challenges were identified for mid-air interaction methods.…”