Real-time photorealistic virtualized reality interface for remote mobile robot control

Abstract

The task of teleoperating a robot over a wireless video link is known to be very difficult. Teleoperation becomes even more difficult when the robot is surrounded by dense obstacles, or speed requirements are high, or video quality is poor, or wireless links are subject to latency. Due to high quality lidar data, and improvements in computing and video compression, virtualized reality has the capacity to dramatically improve teleoperation performance -even in high speed situations that were formerly impossible. In this paper, we demonstrate the conversion of dense geometry and appearance data, generated on-the-move by a mobile robot, into a photorealistic rendering database that gives the user a synthetic exterior line-of-sight view of the robot including the context of its surrounding terrain. This technique converts remote teleoperation into line-of-sight remote control. The underlying metrically consistent environment model also introduces the capacity to remove latency and enhance video compression. Display quality is sufficiently high that the user experience is similar to driving a video game where the surfaces used are textured with live video.

Exploring customer demands for a new assistive robot with visual servo technology for the housing sector

“…The purchasing motives in 12 measurement criteria questions and the preferences in 9 measurement criteria questions were identified from past studies [16]. A survey questionnaire was developed to collect potential consumers' purchasing motives and customer preferences.…”

Natural Interaction Design of a Humanoid Robot

“…In our implementation, no specific precautions were taken to minimize the effect of the network latency's variation when executing user commands and displaying perceived information. As mentioned in Kelly et al (2011), teleoperation becomes even more difficult in environments surrounded by dense obstacles, with network latency and high speed requirements.…”

“…Motion prediction with a velocity-driven dynamic model (Kelly et al, 2011) and wave variables (Munir & Book, 2002) is currently being considered to compensate for latencies that can occur between graphical representation of the robot's state, and perception.…”

Augmented reality user interface for mobile ground robots with manipulator arms

“…However, to transition to a system for field use, more complex sensors such as LIDAR and stereovision combined with sensor fusion techniques could be implemented to extend the utility of this AR UI to unstructured domains, making it suitable for exploration and search-type tasks. In particular, sensor suites that fuse laser scanners and video data to create 3D models of robot surroundings 25,26 are of particular interest.…”