Incorporating the Perception of Visual Roughness into the Design of Mid-Air Haptic Textures

Abstract

Ultrasonic mid-air haptic feedback enables the tactile exploration of virtual objects in digital environments. However, an object's shape and texture is perceived multimodally, commencing before tactile contact is made. Visual cues, such as the spatial distribution of surface elements, play a critical first step in forming an expectation of how a texture should feel. When rendering surface texture virtually, its verisimilitude is dependent on whether these visually inferred prior expectations are experienced during tactile exploration. To that end, our work proposes a method where the visual perception of roughness is integrated into the rendering algorithm of mid-air haptic texture feedback. We develop a machine learning model trained on crowd-sourced visual roughness ratings of texture images from the Penn Haptic Texture Toolkit (HaTT ). We establish tactile roughness ratings for different mid-air haptic stimuli and match these ratings to our model's output, creating an end-to-end automated visuo-haptic rendering algorithm. We validate our approach by conducting a user study to examine the utility of the mid-air haptic feedback. This work can be used to automatically create tactile virtual surfaces where the visual perception of texture roughness guides the design of mid-air haptic feedback.

Multisensory Integration as per Technological Advances: A Review

“…To provide a tactile sensation in midair, ultrasonic phased arrays composed of several speakers (see Figure 1c) can be computer-controlled to emit focused ultrasound over distance (e.g., 20 cm) and enable a person to perceive tactile sensations in mid-air without the need of physical attachments, such as a glove. These tactile sensations can be single or multiple focal points on the hand, 3D shapes (Carter et al, 2013), or textures (Beattie et al, 2020). This unique combination is enabling novel interaction paradigms previously only seen in science fiction movies.…”

Enhancing Ultrasound Haptics with Parametric Audio Effects

“…Freeman et al [12] rendered geometric features at the intersection between the hand and a complex surface as a crude representation of macro-texture elements. Beattie et al [3,4] showed a more nuanced approach to texture generation, extracting features from images and rendering them dynamically in response to hand movement.…”

“…A key characteristic in surface roughness is the physical size and separation of physical elements: macrotexture and micro-texture elements being greater, and less than, 0.2 mm respectively [27]. We point readers towards work by Klatzky and Lederman [27], who discuss texture perception in detail, and Beattie et al [3], who interpret the literature on roughness with a focus on ultrasound haptic perception. The latter is particularly relevant to this paper, since ultrasound haptic stimuli lack the physical elements that contribute to typical roughness perception.…”

“…Tactile roughness can be created with a variety of haptic devices, e.g., vibrotactile actuators [8,16,44], force feedback devices [33,38] and ultrasound haptic devices [3,4,12]. These often induce roughness via temporal rather than spatial cues (e.g., since a vibrotactile actuator cannot move or an ultrasound focal point is too large).…”

Research on Visual Image Texture Rendering for Artistic Aided Design

“…Moreover, 2D texture feature extraction and analysis method of color images is combined to analyze the texture feature of color images, improve the image quality and detection ability of color images, study the 2D texture recognition method of color images, and improve the accurate analysis and 3D feature resolution ability of color multitexture image. In [5], the authors used a combination of macro-and local aspects to obtain multiscale data information for building an image information model. e authors in [6] put forward a method of image segmentation based on the induction and application of multifeature information in remote sensing images.…”