A 3-D Vision-Based Man–Machine Interface For Hand-Controlled Telerobot

Abstract

This paper presents a robust telerobotic system that consists of a real-time vision-based operator hand tracking system (client) and a slave robot (server) which are interconnected through a LAN. The tracking system (1) monitors the operator hand motion and (2) determines its position and orientation which are used to control the slave robot. Two digital cameras are used to monitor a four-ball based feature frame that is held by the operator hand. To determine the 3D position a tracking algorithm based on uncalibrated cameras with weak perspective projection model is used. This allows finding 3D differential position and orientation of operator hand. The features of proposed system are (1) a metric for color matching to discriminate the balls from their background, (2) a uniform and spiral search approach to speedup the detection, (3) tracking in the presence of partial occlusion, (4) consolidate detection by using shape and geometric matching, and (5) dynamic update of the reference colors. The operator can see the effects of the previous motion which enables making the necessary corrections through repetitive operator hand-eye interactions. Evaluation shows that the static and dynamic errors of tracking algorithm are 0.1% and 0.6% for a centered workspace of 20 3 inches 3 that is 40 to 60 inches away from cameras. Running the tracking algorithm on two PCs in parallel allowed (1) a parallel image grabbing delay of 60 ms, (2) a stereo matching delay of 50 ms, and (3) a global refresh rate of 9Hz.

Operation Modes and Control Schemes for a Telerobot with Time Delay

“…The operation mode M could be easy to realize by choosing an autonomy function proposed by Yokokohji [10] in the slave side and using delayed force feedback control in the master side.…”

“…They also used force reflection and object deformation simultaneously to enable the human operator to feel and to observe the influences of interactive force with time delay. Mouhamed et al [10] presents a robust telerobotic system that consists of a realtime vision-based operator hand tracking system and a slave robot which are interconnected through a LAN. Lu et al [11] presented a detailed energy-compensating methodology to improve the performance of haptic rendering by the human operator during interactions.…”

Short-Baseline Binocular Vision System for a Humanoid Ping-Pong Robot

“…Mouhamed et al [23] compute ball centers using boundary pixels. This method needs a threshold to detect ball boundaries, but the threshold often is sensitive to illumination changes.…”

Constraints over Greenhouse Detection using Wireless Sensor Networks

“…As soon as, some of the sensor nodes in a WSN run out of energy, they stop functioning causing progressive deconstruction of the network. Therefore, one of the most stringent limitations that the development of a WSN faces is that of power consumption [1]. Due to current technological advances, the development of materials for tiny and low cost sensors became technically and economically reasonable.…”