More and more network cameras are now working over distributed networks, offering the capability of remote intelligent video surveillance. In this paper, we bring forward an original particle filter tracking algorithm named labelled particle filter which describes each image patch with a binary label. Based on the imaging theory of thermography, moving objects, such as pedestrians and automobiles, usually have higher intensities compared with the background in a gray-level pseudocolor mode. Thus an image patch can be classified into two categories according to its intensity distribution, and we can use a one-bit binary label, positive or negative, to describe the attribute of image patch. Therefore, the candidate target template is established only if the label of candidate target matches the label of reference target, and the computational complexity is reduced consequently. Experiments are conducted to show that the proposed algorithm can handle real-time object tracking with less time cost while maintaining high tracking accuracy.