Refined methods for creating realistic haptic virtual textures from tool-mediated contact acceleration data

Abstract

Dragging a tool across a textured object creates rich high-frequency vibrations that distinctly convey the physical interaction between the tool tip and the object surface. Varying one's scanning speed and normal force alters these vibrations, but it does not change the perceived identity of the tool or the surface. Previous research developed a promising data-driven approach to embedding this natural complexity in a haptic virtual environment: the approach centers on recording and modeling the tool contact accelerations that occur during real texture interactions at a limited set of force-speed combinations. This paper aims to optimize these prior methods of texture modeling and rendering to improve system performance and enable potentially higher levels of haptic realism. The key elements of our approach are drawn from time series analysis, speech processing, and discrete-time control. We represent each recorded texture vibration with a low-order autoregressive moving-average (ARMA) model, and we optimize this set of models for a specific tool-surface pairing (plastic stylus and textured ABS plastic) using metrics that depend on spectral match, final prediction error, and model order. For rendering, we stably resample the texture models at the desired output rate, and we derive a new texture model at each time step using bilinear interpolation on the line spectral frequencies of the resampled models adjacent to the user's current force and speed. These refined processes enable our TexturePad system to generate a stable and spectrally accurate vibration waveform in real time, moving us closer to the goal of virtual textures that are indistinguishable from their real counterparts. ABSTRACTDragging a tool across a textured object creates rich high-frequency vibrations that distinctly convey the physical interaction between the tool tip and the object surface. Varying one's scanning speed and normal force alters these vibrations, but it does not change the perceived identity of the tool or the surface. Previous research developed a promising data-driven approach to embedding this natural complexity in a haptic virtual environment: the approach centers on recording and modeling the tool contact accelerations that occur during real texture interactions at a limited set of force-speed combinations. This paper aims to optimize these prior methods of texture modeling and rendering to improve system performance and enable potentially higher levels of haptic realism. The key elements of our approach are drawn from time series analysis, speech processing, and discrete-time control. We represent each recorded texture vibration with a low-order auto-regressive moving-average (ARMA) model, and we optimize this set of models for a specific tool-surface pairing (plastic stylus and textured ABS plastic) using metrics that depend on spectral match, final prediction error, and model order. For rendering, we stably resample the texture models at the desired output rate, and we derive a new texture model at each time step ...

Data-driven modeling of isotropic haptic textures using frequency-decomposed neural networks

“…Using the same modeling and synthesis methods, they obtained moderately high realism ratings (M 65.4, SD 19 in a 0-100 scale) in a user study. Their model was further refined in [15] by using an ARMA (autoregressive moving average) model to better handle the weak stationary nature of manually captured acceleration data with a greatly reduced number of coefficients. For synthesis, the linear spectral frequencies are interpolated between adjacent ARMA models to ensure the stability of the interpolated model.…”

A haptic texture database for tool-mediated texture recognition and classification

“…When displaying the recorded acceleration signal with a vibrotactile actuator as explained in Section I, the direction of the displayed vibration is not important [15]. For this reason, we use the DFT321 algorithm proposed by Landin et al in [15], which is also used, e.g., in [5], [12], [13] to reduce the dimensionality of the recorded signal. The DFT321 algorithm combines the three-axis acceleration signal into a one-dimensional signal while preserving important temporal and spectral properties.…”

“…Different scan parameters during the recording of haptic textures cause changes in the temporal and spectral properties of the signals [8], [13]. The exploration velocity, the applied force, the aforementioned tip mounted on the stylus, its shape, or the inclination with which the stylus is stroked over the surface may heavily influence signal properties and, thus, also extracted features used for recognition or classification.…”

“…Note that the purpose of the provided recordings is especially the identification of scan-invariant features. This is in contrast to the HaTT, where the focus is on the rendering and display of textures using texture models similar to the models proposed in [12], [13].…”

Dynamic simulation of tool-mediated texture interaction

“…Previous data-driven texture modeling work in our lab [9,10,11] has treated the tool-texture-hand system as a black box with no connection to the underlying physics. The tool vibrations measured at each recorded combination of normal force and scanning speed were fit with an LPC or ARMA model, and these models were driven with white noise to generate spectrally appropriate vibration waveforms during a user interaction.…”