Investigations on collaborative remote control of virtual robotic manipulators by using a Kinect v2 sensor

Abstract

The present work deals with design aspects and testing results of a novel solution regarding the remote control of two virtual robotic manipulators by using hands motion tracking and palms gesture recognition. The spatial position and orientation of the user© s left and right hands© wrists are interpreted by a custom built software application and then forwarded as positioning data and gripper status to a V-Rep server running the simulation scene. The goal of the presented setup is for the user to be able to manipulate objects in the scene in a collaborative way, i.e. pick one object with the first robot and give it to the second one for elsewhere depositing. The real-life applications for this approach are to provide a framework for simplified operator training and programming of robots used in pick-and-place or assembly operations, and also in healthcare for assisting patients in handcoordination and mobility recovery.