In terms of fatalities, Malaysia ranks third among ASEAN countries. Every year, there is an increase in accidents and fatalities. The state of the road is one factor contributing to near misses. A near miss is an almost-caused accident, an unplanned situation that could result in injury or accidents. The Majlis Bandar Pulau Pinang (MBPP) has installed 1841 closed-circuit television (CCTV) cameras around Penang to monitor traffic and track near miss incidents. When installing CCTVs, the utilisation of video allows resources to be used and optimised in situations when maintaining video memories is difficult and costly. Highways, industrial regions, and city roads are the most typical places where accidents occur. Accidents occurred at 200 per year on average in Penang from 2015 to 2017. Near misses are what create accidents. One of the essential factors in vehicle detection is the “near miss.” In this study, You Only Look Once version 3 (YOLOv3) and Faster Region-based Convolutional Neural Network (Faster RCNN) are used to solve transportation issues. In vehicle detection, a faster RCNN was used. Bird’s Eye View and Social Distancing Monitoring are used to detect the only vehicle in image processing and observe how near misses occur. This experiment tests different video quality and lengths to compare test time and error detection percentage. In conclusion, YOLOv3 outperforms Faster RCNN. In high-resolution videos, Faster RCNN outperforms YOLOv3, while in low-resolution videos, YOLOv3 outperforms Faster RCNN.