Abstract. Smart video analytics algorithms can be embedded within surveillance sensors for fast in-camera processing. This paper presents a DSP embedded video analytics system for object and people tracking, using a PTZ camera. The tracking algorithm is based on adaptive template matching and it employs a novel Sum of Weighted Absolute Differences. The video analytics is implemented on the DSP board DM6437 EVM and it automatically controls the PTZ camera, to keep the target central to the field of view. The EVM is connected to the network and the tracking algorithm can be remotely activated, so that the PTZ enhanced with the DSP embedded video analytics becomes a smart surveillance sensor. The system runs in real-time and simulation results demonstrate that the described SWAD outperforms other template matching measures in terms of efficiency and accuracy.