A Stereo-Panoramic Telepresence System for Construction Machines

Abstract

Working machines in construction sites or emergency scenarios can operate in situations that can be dangerous to an operator both\ud for direct control of the machine or teleoperation in co-presence. Remote operation has been typically hindered by limited sense\ud of presence of the operator in the remote of the environment to due the reduced field of view of cameras. Starting from these\ud considerations we are introducing a novel real-time panoramic telepresence system for construction machines. This system does\ud allow fully immersive operations in critical scenarios while keeping the operator in a safe location at moderate distance from the\ud construction site. An omnidirectional stereo vision head mounted over the machine acquires and streams data to the operator with a\ud streaming technique that focuses on the current direction of site of the operator. According to the motion of the operators head, the\ud telepresence system will determine the involved cameras to use and the appropriate region of interest in cameras. The operator uses\ud a head-mounted display to experience the remote site also with the possibility to view digital information overlaid to the remote\ud scene as a type of augmented reality. The paper addresses the design and architecture of the system starting from the vision system\ud and then proceeding to the immersive visualizatio

Latency impact on Quality of Experience in a virtual reality simulator for remote control of machines

“…Therefore, increased efforts should be allocated to understanding the QoE, so that the new technology is not inducing negative perceived user experiences, discomfort or even simulator sickness. Furthermore, low latency can enable services and applications with an intensive interaction component, such as gaming or remote control of professional equipment [16], which will increase the cognitive load on the user. Although research in this field has been ongoing for some time, the rapid technological development and increasing availability of immersive low latency user environments make additional research efforts necessary and more relevant than ever before.…”

An Augmented Reality Based Human-Robot Interaction Interface Using Kalman Filter Sensor Fusion

“…In Reference [10], authors developed an augmented reality-aided tele-presence system for robot manipulation for the usage of industrial manufacturing, where a robot was successfully controlled remotely to pick a known object from a starting position and placing it in different target positions. In Reference [11], a telepresence system that allows the immsersive remote control of a working machine was developed, where an omnidirectional stereo vision head mounted on the machine to interact with the operator who wears a head-mounted display. In Reference [12], the authors developed a novel wearable controller with haptic function which is suitable for various robotic platforms.…”

Joint effects of depth-aiding augmentations and viewing positions on the quality of experience in augmented telepresence

“…In particular, immersive telepresence systems for vehicle operation are fast becoming a reality in industrial applications [9,10], thanks to the push towards 5G low-latency communications. AR is likewise gaining traction as a method for assisting operators in performing specialized tasks [5,6], leading to Augmented Telepresence applications for industry, in use cases like the remote operation of truck mounted forestry cranes shown in Fig.…”

“…Based on the aforementioned cases and ongoing research, Head-Mounted Display (HMD) based AT has potential for improving user Quality of Experience (QoE) in industrial applications. However, if AR assistance is employed in industrial immersive telepresence systems as demonstrated in [9], then the operator QoE and task performance may be affected by the overlap of AR and stereoscopy in unknown ways to an unknown extent. Therefore, studies are required to address the impacts of AR and stereoscopy on operator QoE.…”