Abstract-We describe a general methodology for tracking 3-dimensional objects in monocular and stereo video that makes use of GPU-accelerated filtering and rendering in combination with machine learning techniques. The method operates on targets consisting of kinematic chains with known geometry. The tracked target is divided into one or more areas of consistent appearance. The appearance of each area is represented by a classifier trained to assign a class-conditional probability to image feature vectors. A search is then performed on the configuration space of the target to find the maximum likelihood configuration. In the search, candidate hypotheses are evaluated by rendering a 3D model of the target object and measuring its consistency with the class probability map. The method is demonstrated for tool tracking on videos from two surgical domains, as well as in a human hand-tracking task.

International audienceThe tracking of surgical instruments o fers interesting possibilities for the development of high-level commands for robotic camera holders in laparoscopic surgery. We have developed a new method to detect instruments in laparoscopic images which uses information on the 3D position of the insertion point of an instrument into the abdominal cavity. This information strongly constrains the search for the instrument in each endoscopic image. Hence, the instrument can be detected in near real-time using shape considerations. Early results on laparoscopic images show that the method is rapid and robust in the presence of partial occlusion and smoke. Our first experiment on a cadaver validates our approach and shows encouraging result