This paper presents a technique for a robust tracking of the target object by integration of the vision-based disturbance observer with bilateral haptic system. As for the motion navigation task, the bilateral control of master-slave system is combined with the vision-based force compliance controller. The disturbance of the modelling error that occurs in the integration of the system will be compensated by the proposed vision-based disturbance observer. The concept of control structure between the integration of bilateral manipulator, vision-based force compliance controller and the proposed visionbased disturbance observer are described. Two experiments were conducted to compare the result of without and with the proposed integration method. From the experimental results, the robustness of the proposed system is confirmed.