Traffic surveillance is a system used to observe the traffic congestion of a specific area. The surveillance system involves detection of moving vehicles, counting the number of vehicles and the classification of the detected vehicles. Recognition of the moving vehicles is the initial step which can be carried out either in each and every frame based on background modeling with discrete cosine transform (DCT). The proposed method is an efficient approach for vehicle segmentation based on texture analysis, a 2D discrete cosine transform (DCT) which is utilized to extract texture features in each image block. By this method the foreground object is detected by eliminating the background, vehicles are recognized based on their size and highlighted by drawing a bounding box around them. The actual count of the vehicles is estimated by counting the number of bounding box present in the video. Further the detected vehicles present in the video can be classified into car, bike, truck etc. based on the SVM classifiers. Support vector machine (SVM) are supervised learning tools which is applied for data classification and regression, It maps the training samples that are the points in features space into different categories which are clearly separated with the widest gap in between them. The proposed system can be used to conduct a survey and allow the users to monitor the condition of traffic flow by counting the number of vehicles passing through a specific location.