Abstract-To solve the problem of reliability and real time in visual perception of the mobile manipulator, a visual based information collection system integrated edge detection algorithm and tracking algorithm is proposed. In the vision system, an adaptable threshold edge detection algorithm based on the small world network is presented to improve the reliability and accuracy of edge detection and image segmentation, and the Meanshift algorithm is used to track the mobile objective in real time. Firstly, the edge detection algorithm based on small world phenomenon can adapt the threshold from the global to the local and it improves the ability of the mobile manipulator to find the moving object. Secondly, the edge detection results are used to initialize the target area of Meanshift algorithm, which then tracks the moving objective and extracts parameters' information automatically and real time. The proposed vision system not only meets the need of a mobile manipulator operating on moving objectives but also provides an effective method to find and track objectives automatically.