No abstract

Abstract-In a previous paper [3], we proposed a new way to achieve visual servoing. Rather than minimizing the error between the position of two set of geometric features, we proposed to maximize the mutual information shared by the current and desired images. This leads to a new information theoretic approach to visual servoing. Mutual information is a well known alignment function. Thanks to its robustness toward illumination variations, occlusions and multi modality, it has been widely used in medical applications for alignment as well as in general tracking problems. Despite those previous works, no highlight has been given on the problem of Hessian computation that yields, in the case of common approximations, to divergence of the optimization process. In this paper we focus on the need of computing the second order derivative of the mutual information in visual servoing. Experiments on a 6 dof robot demonstrates the significance of this work on visual servoing tasks.

In this paper we present a direct tracking approach that uses Mutual Information (MI) as a metric for alignment. The proposed approach is robust, real-time and gives an accurate estimation of the displacement that makes it adapted to augmented reality applications. MI is a measure of the quantity of information shared by signals that has been widely used in medical applications. Since then, and although MI has the ability to perform robust alignment with illumination changes, multi-modality and partial occlusions, few works propose MI-based applications related to object tracking in image sequences due to some optimization problems. In this work, we propose an optimization method that is adapted to the MI cost function and gives a practical solution for augmented reality application. We show that by refining the computation of the Hessian matrix and using a specific optimization approach, the tracking results are far more robust and accurate than the existing solutions. A new approach is also proposed to speed up the computation of the derivatives and keep the same optimization efficiency.To validate the advantages of the proposed approach, several experiments are performed. The ESM and the proposed MI tracking approaches are compared on a standard dataset. We also show the robustness of the proposed approach on registration applications with different sensor modalities: map versus satellite images and satellite images versus airborne infrared images within different AR applications.