Background Digital rectal examination is a difficult examination to learn and teach because of limited opportunities for practice; however, the main challenge is that students and tutors cannot see the finger when it is palpating the anal canal and prostate gland inside the patients. Objective This paper presents an augmented reality system to be used with benchtop models commonly available in medical schools with the aim of addressing the problem of lack of visualization. The system enables visualization of the examining finger, as well as of the internal organs when performing digital rectal examinations. Magnetic tracking sensors are used to track the movement of the finger, and a pressure sensor is used to monitor the applied pressure. By overlaying a virtual finger on the real finger and a virtual model on the benchtop model, students can see through the examination and finger maneuvers. Methods The system was implemented in the Unity game engine (Unity Technologies) and uses a first-generation HoloLens (Microsoft Inc) as an augmented reality device. To evaluate the system, 19 participants (9 clinicians who routinely performed digital rectal examinations and 10 medical students) were asked to use the system and answer 12 questions regarding the usefulness of the system. Results The system showed the movement of an examining finger in real time with a frame rate of 60 fps on the HoloLens and accurately aligned the virtual and real models with a mean error of 3.9 mm. Users found the movement of the finger was realistic (mean 3.9, SD 1.2); moreover, they found the visualization of the finger and internal organs were useful for teaching, learning, and assessment of digital rectal examinations (finger: mean 4.1, SD 1.1; organs: mean 4.6, SD 0.8), mainly targeting a novice group. Conclusions The proposed augmented reality system was designed to improve teaching and learning of digital rectal examination skills by providing visualization of the finger and internal organs. The initial user study proved its applicability and usefulness.

Electrovibration technology has the potential for seamless integration into ordinary smartphones and tablets to provide programmable haptic feedback. The aim of this work is to seek effective ways to improve 3D perception of visual objects rendered on an electrovibration display. Utilizing a gradient-based algorithm, we first investigated whether rendering only lateral frictional force on an electrovibration display improves 3D shape perception compared to doing the same using a force-feedback interface. We observed that although users do not naturally associate electrovibration patterns to geometrical shapes, they can map patterns to shapes with moderate accuracy if guidance or context is given. Motivated by this finding, we generalized the gradient-based rendering algorithm to estimate the surface gradient for any 3D mesh and added an edge detection algorithm to render sharp edges. Then, we evaluated the advantages of our algorithm in a user study and found that our algorithm can notably improve the performance of 3D shape recognition when visual information is limited.

With the introduction of variable friction displays, either based on ultrasonic or electrovibration technology, new possibilities have emerged in haptic texture rendering on flat surfaces. In this work, we propose a data-driven method for realistic texture rendering on an electrovibration display. We first describe a motorized linear tribometer designed to collect lateral frictional forces from textured surfaces under various scanning velocities and normal forces. We then propose an inverse dynamics model of the display to describe its output-input relationship using nonlinear autoregressive neural networks with external input. Forces resulting from applying a pseudo-random binary signal to the display are used to train each network under the given experimental condition. In addition, we propose a two-step interpolation scheme to estimate actuation signals for arbitrary conditions under which no prior data have been collected. A comparison between real and virtual forces in the frequency domain shows promising results for recreating virtual textures similar to the real ones, also revealing the capabilities and limitations of the proposed method. We also conducted a human user study to compare the performance of our neural-network-based method with that of a record-and-playback method. The results showed that the similarity between the real and virtual textures generated by our approach was significantly higher.

Background Performing physiotherapy exercises in front of a physiotherapist yields qualitative assessment notes and immediate feedback. However, practicing the exercises at home lacks feedback on how well patients are performing the prescribed tasks. The absence of proper feedback might result in patients performing the exercises incorrectly, which could worsen their condition. We present an approach to generate performance scores to enable tracking the progress by both the patient at home and the physiotherapist in the clinic. Objective This study aims to propose the use of 2 machine learning algorithms, dynamic time warping (DTW) and hidden Markov model (HMM), to quantitatively assess the patient’s performance with respect to a reference. Methods Movement data were recorded using a motion sensor (Kinect V2), capable of detecting 25 joints in the human skeleton model, and were compared with those of a reference. A total of 16 participants were recruited to perform 4 different exercises: shoulder abduction, hip abduction, lunge, and sit-to-stand exercises. Their performance was compared with that of a physiotherapist as a reference. Results Both algorithms showed a similar trend in assessing participant performance. However, their sensitivity levels were different. Although DTW was more sensitive to small changes, HMM captured a general view of the performance, being less sensitive to the details. Conclusions The chosen algorithms demonstrated their capacity to objectively assess the performance of physical therapy. HMM may be more suitable in the early stages of a physiotherapy program to capture and report general performance, whereas DTW could be used later to focus on the details. The scores enable the patient to monitor their daily performance. They can also be reported back to the physiotherapist to track and assess patient progress, provide feedback, and adjust the exercise program if needed.

No abstract