Visuo-Haptic Mixed Reality Simulation Using Unbound Handheld Tools

Abstract

Visuo-haptic mixed reality (VHMR) adds virtual objects to a real scene and enables users to see and also touch them via a see-through display and a haptic device. Most studies with kinesthetic feedback use general-purpose haptic devices, which require the user to continuously hold an attached stylus. This approach constrains users to the mechanical limits of the device even when it is not needed. In this paper, we propose a novel VHMR concept with an encountered-type haptic display (ETHD), which consists of a precision hexapod positioner and a six-axis force/torque transducer. The main contribution is that the users work with unbound real-life tools with tracking markers. ETHD’s end-effector remains inside the virtual object and follows the tooltip to engage only during an interaction. We have developed a simulation setup and experimentally evaluated the relative accuracy and synchronization of the three major processes, namely tool tracking, haptic rendering, and visual rendering. The experiments successfully build-up to a simple simulation scenario where a tennis ball with a fixed center is deformed by the user.

A comprehensive study to learn the impact of augmented reality and haptic interaction in ultrasound-guided percutaneous liver biopsy training and education

“…As a result, the trainees receive haptic feedback from the needle which is replicated by the haptic device and the syringe. Sometimes electric actuators are used to develop custom haptic interfaces along with a hexapod design (Aygün et al 2020). The MURAB project has introduced a revolutionary method for conducting cancer biopsies in collaboration with KUKA Robotics and SIEMENS.…”

Integrating Virtual, Mixed, and Augmented Reality into Remote Robotic Applications: A Brief Review of Extended Reality-Enhanced Robotic Systems for Intuitive Telemanipulation and Telemanufacturing Tasks in Hazardous Conditions

“…By creating an immersive and interactive MR environment, virtual workpieces can be generated in the userʹs space. When combined with Virtual Fixture (VF) technology, this environment provides users with force feedback and guidance, improving the precision of robot movements and preventing accidental collisions [168,169].…”

Review on Needle Insertion Haptic Simulation

“…In [15], the authors developed a custom interface based on electric actuators and a hexapod design (see Fig. 4).…”

“…Fig. 4: Simulator using a custom haptic device based on an electric actuated hexapod [15] In epidural procedures, practitioners handle a needle mounted on a syringe. Epidural anesthesia is a blind procedure as practitioners cannot see through the human body and do not use any ultrasound probes on daily use.…”