Robust colour and range sensing for robotic applications using a stereoscopic light stripe scanner

et al. 2004

Int. J. Human. Robot.

Sensing is a key element for any intelligent robotic system. This paper describes the current progress of a project in the Intelligent Robotics Research Center at Monash University that has the aim of developing a synergistic set of sensory systems for a humanoid robot. Currently, sensing modes for colour vision, stereo vision, active range, smell and airflow are being developed in a size and form that is compatible with the humanoid appearance. Essential considerations are sensor calibration and the processing of sensor data to give reliable information about properties of the robot's environment. In order to demonstrate the synergistic use of all of the available sensory modes, a high level supervisory control scheme is being developed for the robot. All time-stamped sensor data together with derived information about the robot's environment are organized in a blackboard system. Control action sequences are then derived from the blackboard data based on a task description. The paper presents details of each of the robot's sensory systems, sensor calibration, and supervisory control. Results are also presented of a demonstration project that involves identifying and selecting mugs containing household chemicals. Proposals for future development of the humanoid robot are also presented.

Section: Visionmentioning

confidence: 99%

Section: Visionmentioning

confidence: 99%

Multi-Sensory Synergies in Humanoid Robotics

Russell

et al. 2004

Int. J. Human. Robot.

“…Since the intended application is to support object modeling and tracking (Taylor and Kleeman 2001), this implicit registration of color and range is highly desirable. The system described in this work was first proposed in Taylor, Kleeman, and Wernersson (2002), and is extended here by providing a more rigorous treatment with respect to measurement noise.…”

Section: Introductionmentioning

confidence: 99%

Stereoscopic Light Stripe Scanning: Interference Rejection, Error Minimization and Calibration

The International Journal of Robotics Research

2004

This paper addresses the problem of rejecting interference due to secondary specular reflections, cross-talk and other mechanisms in an active light stripe scanner for robotic applications. Conventional scanning methods control the environment to ensure the brightness of the stripe exceeds that of all other features. However, this assumption is likely to be violated for a robot operating in an uncontrolled environment. Robust scanning methods already exist, but suffer from problems including assumed scene structure, acquisition delay, lack of error recovery, and incorrect modeling of measurement noise. We propose a robust technique that overcomes these problems, using two cameras and knowledge of the light plane orientation to disambiguate the primary reflection from spurious measurements. Unlike other robust techniques, our validation and reconstruction algorithms are optimal with respect to sensor noise. Furthermore, we propose a procedure to calibrate the system using measurements of an arbitrary non-planar target, providing robust validation independently of ranging accuracy. Finally, our robust techniques allow the sensor to operate in ambient indoor light, allowing color and range to be implicitly registered. An experimental scanner demonstrates the effectiveness of the proposed techniques. Source code and sample data are provided in the multimedia extensions.

“…To overcome these issues, we proposed the stereoscopic light stripe scanner shown in Figure 3 [17]. On corresponding stereo scanlines, measurements of a point X on the light plane satisfy a linear relationship (in projective space) of the form L x = H R x, where the transformation H is related to the stereo geometry and position of the light plane.…”

Section: A Stereoscopic Light Stripe Scanningmentioning

confidence: 99%

“…Our work develops the perception and control skills necessary to perform simple manipulations (such as pouring a drink) with minimum a priori knowledge in an unstructured domestic environment, which forms the basis of important applications such as assisting the elderly and disabled. The proposed framework is based on visual sensing, and integrates our previous work in 3D range scanning [17], data-driven object classification and modelling [16], multimodal tracking [15], and visual servoing [13]. To perform tasks with maximum flexibility, our novel methods address the following key challenges:…”

Section: Introductionmentioning

confidence: 99%

Integration of robust visual perception and control for a domestic humanoid robot

2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566)

Self Cite

Abstract-This paper describes a complete vision-based framework that enables a humanoid robot to perform simple manipulations in a domestic environment. Our system emphasizes autonomous operation with minimal a priori knowledge in an unstructured environment, with robustness to visual distractions and calibration errors. For each new task, the robot first acquires a dense 3D image of the scene using our novel stereoscopic light stripe scanner that rejects secondary reflections and cross-talk. A data-driven analysis of the range map identifies and models simple objects using geometric primitives. Objects are reliably tracked through clutter and occlusions by exploiting multimodal cues (colour, texture and edges). Finally, manipulations are performed by controlling the end-effector using a hybrid position-based visual servoing scheme that fuses visual and kinematic measurements and compensates for calibration errors. Two domestic tasks are implemented to evaluate the performance of the framework: identifying and grasping a yellow box without any prior knowledge of the object, and pouring rice from an interactively selected cup into a bowl.