Evaluation of Communication and Human Response Latency for (Human) Teleoperation

Abstract

<p>We previously introduced a novel mixed reality (MR) teleguidance system, human teleoperation [1,2], in which a human (expert) leader and a human (novice) follower are tightly coupled through MR and haptics for applications such as tele-ultrasound. In this paper, a communication system suitable for human teleoperation is presented and characterized in various network conditions, over Ethernet, Wi-Fi, 4G LTE, and 5G. To study all types of latency in the system, the human response time is additionally characterized through step response tests with 11 volunteers. The step responses were obtained by tracking the position and force of the human hand in response to a change in the MR target.</p> <p>The round-trip communication latency is 40+/-10 ms over 5G, and down to 1+/-0.6 ms over Ethernet for typical throughputs. The human response time to a step change in position depends on the step magnitude, but is 485-535 ms, while the reaction time for forces is 150-200 ms. Both lags are decreased when tracking smooth motions. Thus, we demonstrate that the system is network agnostic and can achieve good teleoperation performance and secure, fast communication in appropriate network conditions. The presented tools and concepts are applicable to any high-performance teleoperation system, for example for remote surgery.</p>

Robust Object Pose Tracking for Augmented Reality Guidance and Teleoperation

“…To compute lag between the signals, the Polaris data was read in programmatically while the same program periodically exchanged timestamps over a WebSocket connection with the HoloLens, as described previously [58]. This enabled clock synchronization between the HoloLens and the PC connected to the Polaris.…”