Human-as-a-Robot Performance in Augmented Reality Teleultrasound

Abstract

<p> <strong>Purpose:</strong> In “human teleoperation” [1], augmented reality (AR) and haptics are used to tightly couple an expert leader to a human follower. To determine the feasibility of human teleoperation, we quantify the abil- ity of humans to track a position and/or force trajectory via AR cues. The human response time, precision, frequency response, and step response were characterized, and several rendering methods were compared. </p> <p><strong>Methods:</strong> Volunteers (n=11) performed a series of tasks as the fol- lower in our human teleoperation system. The tasks involved tracking pre-recorded series of motions and forces, each time with a different rendering method. The order of tasks and rendering methods was ran- domized to avoid learning effects and bias. The volunteers then performed a series of frequency response tests and filled out a questionnaire. </p> <p><strong>Results:</strong> Rendering the full ultrasound probe as a position target with an error bar displaying force led to the best position and force tracking. Following force and pose simultaneously was more difficult but did not lead to significant performance degradation versus following one at a time. On average, subjects tracked positions, orientations, and forces with rms tracking errors of 6.2 ± 1.9 mm, 5.9 ± 1.9˚, 1.0 ± 0.3 N, steady-state errors of 2.8 ± 2.1 mm, 0.26 ± 0.2 N, and lags of 345.5 ± 87.6 ms respectively. Performance decreased with input frequency, until the person could no longer follow, depending on the input amplitude. </p> <p><strong>Conclusion:</strong> This paper characterizes human tracking ability in aug- mented reality human teleoperation, which shows the system’s feasi- bility and good performance, and is important for designing future human computer interfaces using augmented and virtual reality. </p>

Evaluation of Communication and Human Response Latency for (Human) Teleoperation

“…Human trials were carried out in [31] to investigate this ability of humans to act as a robot in tracking an input MR signal, showing promising performance. However, these tests were performed with the expert and follower sides connected over WiFi, on a fast network.…”

“…To perform these tests, the prototype developed in [1] was improved to take direct force input from the expert, and a dummy ultrasound probe was designed for the follower, including 6 axis force/torque sensing and 6-DOF position and orientation (pose) tracking (Section II-B). A novel visual control system for the forces was developed, as described in Section II-C. Further rigorous tests of the visual force control and human tracking ability are presented in [31].…”

“…Similarly, the measured pose can be compared to the desired one to produce a feedback signal, or it can be used in conjunction with the measured force to estimate the mechanical tissue impedance to feed back to the expert's haptic device. In [31] and Section III-C, the measurements are used to characterize human performance in the system.…”

“…This can be installed between a 3D printed shell and the ultrasound probe as done in [14][47] [48][49] [50]. For the tests presented here and in [31], it was instead installed at the tip of a 3D printed dummy ultrasound probe to ensure best possible accuracy. The instrumented dummy ultrasound probe is shown in Fig.…”

“…In some lighting conditions, the virtual probe can occlude the real one, leading to increased position error. Thus, the effectiveness of a full probe rendering was compared to a scheme in which the central part of the virtual probe was removed in [31]. Additionally, the full probe's opacity can be adjusted.…”