In this paper, we present a semi real-time vehicle tracking algorithm to determine the speed of the vehicles in traffic from traffic cam video. The results of this work can be used for traffic control, security and safety both by government agencies and commercial organizations. In this paper a method is described for tracking moving objects from a sequence of video frame. This method is implemented by using optical flow (Horn-Schunck)and (Lucas-Kanade) in mat lab and Simulink. It has a variety of uses, some of which are: human computer interaction, security and surveillance, video communication and compression, augmented reality, traffic control, medical imaging and video editing. Segmentation is performed to detect the object after reducing the noise from that scene. The object is tracked by plotting a rectangular bounding box around it in each frame. The velocity of the object is determined by calculating the distance that the object moved in a sequence of frames with respect to the frame rate that the video is recorded. Comparison and performance analysis of algorithms based on psnr and average angular error is done.