In this paper, we combine rigid motion-based tracking algorithms and non-linear identification methods for automatic detecting and tracking vehicles' trajectory in roadways. In addition, we introduce the concept of the angle spectrum for determining the deviation of a vehicle trajectory from the ideal trace, provided by surveyor engineers. Motion-based tracking is implemented through frame differencing and advanced non-linear convolution filters such as the morphological opening by reconstruction. However, motion based tracking suffers from noise, occlusions and the fact that the detected moving region may contains more than one foreground objects (e.g., a vehicle approach another vehicle). For this reason, a neural network-based classification scheme is adopted in this paper for identifying foreground/background objects. The neural network models the colour and texture properties of the detected moving objects. Fusion algorithm are then exploited which it combine the output of the neural network classifier and the output of the motionbased tracking for efficiently detecting the vehicles trajectory. In the following, we introduce the concept of the angle spectrum which estimates the deviation between two curves, i.e., the vehicle trajectory and the ideal trace. The angle spectrum is computed through quantization of the polar coordinate space, adopted for the curve representation along with novel matching schemes. Experimental results are presented, which indicate the performance of the proposed method in real file environments.