Multisensory Pseudo‐Haptics for Rendering Manual Interactions with Virtual Objects

Abstract

Over the past decade, advances in optics, displays, graphics, tracking, environment mapping, and audio have revolutionized technologies for extended reality (XR). The scope of XR has exploded in recent years, with applications spanning education, marketing, and remote work, as well as training for medicine, industry, and military. [1] All-day wearable XR displays are likely to reinvent computer interfaces in ways that rival the smartphone and personal computer, dramatically changing the way we interact with both the digital and physical worlds, as well as with other people. As we move toward a future in which seeing and hearing virtual objects is commonplace, we must also consider another important sensory aspect─touch.The sensation of touch is critical to our ability to interact with objects in the virtual world just as it is in the physical world, yet there remain significant challenges in synthesizing believable haptic interactions. The earliest haptic device designers proposed that for interactions to feel realistic, the haptic device must make free space feel free and must render stiff virtual objects. [2] These objectives led to the development of probe-based devices that exhibited low inertia and little to no backlash in their transmission mechanisms, and required anchoring to desktop surfaces so that world-grounded stiffnesses and resistance could be rendered to the user. XR haptic device designers are presented with yet more challenges. XR devices not only need to meet free space and stiffness criteria but must do so in an ungrounded, low encumbrance manner. So far, most efforts have focused on wireless haptic controllers, [3][4][5] fingertip displays, [6] and haptic gloves. [7] While these devices address the free-space consideration, they often cannot render virtual stiffnesses and, critically, prevent or degrade concurrent interaction with physical objects in all-day XR contexts. Soft, skin-like materials and devices may show promise in this way, imposing negligible physical burden on users, while delivering reliable sensations and sufficient forces to the skin [8][9][10] ; however, much of this technology is still under development and further from implementation.How can we render virtual stiffnesses without being grounded to the world or encumbering the hands? One method that has

Integration of Physical Reality Objects with Their 3D Models Visualized in Virtual Environment Systems

“…In the field of the touch sense imitation when the user contacts with synthesized virtual environment, active research related to the various aspects of this problem is being conduct-ed. Thus, the paper [7] considers the implementation of tactile sensations during human interaction with virtual objects using the example of pressing buttons. The essence of the approach is to combine visual pseudo-haptic feedback from slowing down the speed of virtual hand movement relative to the real one and tactile pseudo-haptic feedback from effecting on the wrist (squeeze and vibrations) by means of a special bracelet.…”

Multiscale Textile‐Based Haptic Interactions

“…Wearable technology has evolved with the integration of haptic feedback, resulting in enhanced sensory experiences and increased interaction with digital information. [1] In particular, the delivery of tactile cues in wearables has proven instrumental in enhancing user interactions, [2][3][4] increasing perception in virtual environments, [5][6][7][8] augmenting tasks such as robotic teleoperation, [9][10][11][12][13] and facilitating wearable-and prostheticbased rehabilitation. [14][15][16][17][18] This spread of applications underscores the wide interest and investment in the development of information-dense haptic wearables in the research and commercial spaces.…”

“…[13,[27][28][29] To counter this issue of interaction, researchers have trended toward investigating devices worn in less obtrusive areas, albeit with lower densities of mechanoreceptors, such as the forearm. [8,[29][30][31][32] Despite the advancements demonstrated in these prior works, the forearm's limited perceptual acuity has severely limited the development and adoption of a multi-use haptic device worn on the forearm.…”