Visual servoing for micro-manipulation

Abstract

A framework for automated micromanipulation of microscopic objects using visual information is developed and applied to diverse problems in biology and material science. These applications include microdissection of stretched and immobilized DNA molecules using an optical microscope and dynamic studies of crystal structure formation observed with a high power transmission electron microscope. Manipulation in this context refers not only to mechanical positioning or dissection of a specimen, but also to the variation of environmental parameters that elicit a response in the specimen. The common thread between these applications is a collection of computational techniques for image compression, autofocusing, object detection, tracking, and servo l o o p c o n trol. A second feature of our system is a distributed software architecture that allows the proposed dynamic experiments to be conducted remotely over the wide area network. In this context, the visual information processing is used to compensate for dynamic changes and the latencies in the wide area network. This system is now a v ailable over the global Internet.

An ultraprecision six-axis visual servo-control system

“…Using visual feedback from optical microscopes to control micromanipulation processes has been investigated by several researchers [4]- [9] for applications in both the engineering and the biomedical field. For most work done in this area, true realtime visual servoing only involves two DOFs, i.e., and , within the focus plane of the microscope; a natural consequence due to the narrow depth of focus of an optical microscope.…”

Fast autofocus of microscopy images based on Depth-from-Defocus

“…Accordingly, microscope with high amplification and resolution is required for the micromanipulation and microassembly. Microscopic computer vision is of fundamental importance to micromanipulation and microassembly in providing noncontact feedback of microscale geometry, spatial relations and high-level task understanding [2][3][4]. However, because of the unique properties of microscope optics, microscopic computer vision differs significantly from macroscale computer vision.…”

High-speed, wide area, data intensive computing: a ten year retrospective

“…It has already been shown in various collaboratory 3 projects that remote users will gain significant efficiencies through transparent remote access involving widely distributed data intensive computing in almost every case. See, for example, [4], [5], [18], and [19].…”

“…Visual servoing uses information such as object shape change and velocity, obtained by analysis of the video images of the object, to provide the feedback needed for automated control of various microscopic regime instruments. The instruments, in turn, do things like cleaving DNA molecules [4] and controlling the shape of growing micro-crystals [5].…”