Preference-Driven Texture Modeling Through Interactive Generation and Search

Lu, Shihan; Zheng, Mianlun; Fontaine, Matthew C.; Nikolaidis, Stefanos; Culbertson, Heather

doi:10.1109/toh.2022.3173935

Cited by 18 publications

(5 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Tool-mediated haptic texture and sound feedback provide detailed information about the properties of textured surfaces, including roughness, hardness, and slipperiness 3 , 5 , 20 . Our approach is based on the data-driven methodology for modeling and rendering haptic textures in the form of vibrotactile feedback as well as sounds during interactions with stylus-based surfaces.…”

Section: Related Workmentioning

confidence: 99%

“…In our previous work 4 , we formulated a deep spatio-temporal network (DSTN) aimed at data-driven modeling and rendering of haptic textures. Lu et al 20 developed a texture model generator based on Generative Adversarial Networks (GANs), enabling the creation of diverse texture models through Auto-Regressive functions. Their preference-driven interactive texture search approach employs an evolutionary strategy by incorporating feedback from the user’s preferred responses among a set of generated texture models.…”

Section: Related Workmentioning

confidence: 99%

“…Yet, despite these efforts, their performance was not adequate. Recently proposed deep learning-based data-driven approaches, i.e., the deep spatiotemporal network (DSTN) 4 and the deep convolutional generative adversarial network (DCGAN) 20 , have exhibited substantial improvements over previous methods for vibrotactile signal modeling and rendering. Additionally, data-driven modeling based on Wavelet trees is employed for rendering the sounds of tool-surface interactions 5 .…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A multimodal multitask deep learning framework for vibrotactile feedback and sound rendering

Joolee,

Uddin

2024

Sci Rep

View full text Add to dashboard Cite

Data-driven approaches are often utilized to model and generate vibrotactile feedback and sounds for rigid stylus-based interaction. Nevertheless, in prior research, these two modalities were typically addressed separately due to challenges related to synchronization and design complexity. To this end, we introduce a novel multimodal multitask deep learning framework. In this paper, we developed a comprehensive end-to-end data-driven system that encompasses the capture of contact acceleration signals and sound data from various texture surfaces. This framework introduces novel encoder-decoder networks for modeling and rendering vibrotactile feedback through an actuator while routing sound to headphones. The proposed encoder-decoder networks incorporate stacked transformers with convolutional layers to capture both local variability and overall trends within the data. To the best of our knowledge, this is the first attempt to apply transformer-based data-driven approach for modeling and rendering of vibrotactile signals as well as sounds during tool-surface interactions. In numerical evaluations, the proposed framework demonstrates a lower RMS error compared to state-of-the-art models for both vibrotactile signals and sound data. Additionally, subjective similarity evaluation also confirm the superiority of proposed method over state-of-the-art.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A multimodal multitask deep learning framework for vibrotactile feedback and sound rendering

Joolee,

Uddin

2024

Sci Rep

View full text Add to dashboard Cite

show abstract

“…Early pioneering research focused on the use of physics-based simulations to drive tactile rendering for multimedia terminals [30]. The principle of physics-based simulation is to use an AC-driven [31] rendering model to reproduce tactile rendering by simulating the interaction forces between the finger and the touch surface.…”

Section: Physics-based Simulationmentioning

confidence: 99%

A review of enabling tactile rendering on multimedia terminals

Zhang

Zhu

2023

International Conference on Signal Processing, Computer Networks, and Communications (SPCNC 2022)

View full text Add to dashboard Cite

Tactile rendering is cutting-edge technology in the field of human-computer interaction. Tactile rendering technology applied to multimedia terminals has broad application prospects in medicine, education, e-commerce, entertainment, military, and other areas. Multimedia terminal tactile rendering provides tactile interaction between human fingers and multimedia terminals, opens up a new sensory channel for communication between human beings and the virtual world, enhances the realism and immersion of human-computer interaction, and has important analysis and research value. This article provides a comprehensive review of the rationales and research status of three main multimedia terminal tactile rendering methods, introduces the driving methods and common databases, and analyzes the application prospects and future development trends of multimedia terminal tactile rendering.

show abstract

“…A significant recent advancement is the ability to create new textures from texture models. These include predicting vibrations of unmeasured textures [12] [13] [14], texture authoring [15], and texture suggestions based on user pref-erences [16]. In particular, texture generation systems based on user preferences can potentially promote the development of new materials in the surface processing industry [17].…”

mentioning

confidence: 99%

A Simplified Texture Modeling Using a Physical and Perceptual Rule-Based Approach

Tozuka,

Igarashi

2024

IEEE Access

View full text Add to dashboard Cite

In this paper, a method for creating texture models with physical significance using a parametric equalizer (PEQ)-based approach is presented. The model is based on the frequency, amplitude, bandwidth, and noise ratio corresponding to texture profiles. Additionally, the method simplifies the modeling process by enabling the selective omission of imperceptible roughness layers based on human tactile thresholds. The effectiveness of the simplified model was validated through subjective evaluations comprising both absolute and relative assessments. The results confirmed that layers with imperceptible roughness could be excluded without compromising perceived similarity, streamlining the texture modeling process. Regression analysis revealed that PEQ parameters reflect physical interactions in tactile sensations, such as the relationship between texture roughness and vibration amplitude. This study contributes to haptic texture modeling by offering a method that efficiently reproduces actual textures and mirrors the fundamental physical principles of touch. The findings hold promise for applications in texture authoring and material selection, indicating potential advancements in the development of more intuitive and physically transparent texture models.

show abstract

Preference-Driven Texture Modeling Through Interactive Generation and Search

Cited by 18 publications

References 31 publications

A multimodal multitask deep learning framework for vibrotactile feedback and sound rendering

A multimodal multitask deep learning framework for vibrotactile feedback and sound rendering

A review of enabling tactile rendering on multimedia terminals

A Simplified Texture Modeling Using a Physical and Perceptual Rule-Based Approach

Contact Info

Product

Resources

About