Systems must be capable of detecting and tracking autonomous vehicles for intelligent management and control of transportation. Even though several methods are used to create intelligent systems for traffic monitoring, this article explains how to detect and track vehicles using pixel labeling and particle filter algorithms. We suggested a novel technique that segments the image using image segmentation to retrieve the foreground objects. We have divided our proposed model into the following steps: at first, geo-referencing is used to find the exact location; secondly, the images are denoised by using preprocessing; image segmentation is used to separate the background from the foreground; multiple objection detection is performed using the random forest to classify different objects; vehicles are detected through a method called template matching; after this, the vehicles are counted using histogram of oriented gradients (HOG); and after counting, the tracking of vehicles is obtained using particle filter; lastly, the trajectories are predicted by comparing the rectangular centroid of each car against the frame number and using it as a time stamp reference, the last match that the tracking algorithm obtained for each vehicle was recorded and used to estimate the trajectories. Our model outperforms current traffic monitoring approaches in terms of detection and tracking accuracy by 0.87 and 0.92, respectively using the Aerial Car dataset and 0.84 and 0.88, respectively, using the AU-AIR datasets. Vehicle recognition, traffic density detection, traffic flow analysis, and pedestrian route generation are all possible uses for the proposed system. It can transform traffic management and improve overall road safety due to its powerful algorithms and cutting-edge technologies. The system's adaptability makes it a significant asset in modern transportation, from optimizing signal timing to improving pedestrian navigation.INDEX TERMS geo-referencing, random forest, particle filter, vehicle detection, tracking, and trajectories.