Today, movement recognition becomes a highlight in visual analysis. One problem in movement recognition is that existing characteristics of moving objects cannot be applied to recognize moving targets with similar color components and different color structures, which can be easily recognized by human vision. So, in this paper, a novel target tracking method is presented by both color topology and movement features. First, color topology of moving targets is extracted based on analysis of visual mechanism in human vision. In this way, the topology is processed as block diagonal matrix and divided to sub-topological matrices. Then, moving targets are recognized by both similarity of color topological matrices and movement features. Finally, a distributed dynamic tracking method is presented by the division of moving targets because of the huge computation of movement recognition and topological similarity. Experimental results show the high accuracy and real-time of the proposed method.