In view of the current problems of low detection accuracy, poor stability and slow detection speed of intelligent vehicle violation detection systems, this article will use human–computer interaction and computer vision technology to solve the existing problems. First, the picture data required for the experiment is collected through the Bit Vehicle model dataset, and computer vision technology is used for preprocessing. Then, use Kalman filtering to track and study the vehicle to help better predict the trajectory of the vehicle in the area that needs to be detected; finally, use human–computer interaction technology to build the interactive interface of the system and improve the operability of the system. The violation detection system based on computer vision technology has an accuracy of more than 96.86% for the detection of the eight types of violations extracted, and the average detection is 98%. Through computer vision technology, the system can accurately detect and identify vehicle violations in real time, effectively improving the efficiency and safety of traffic management. In addition, the system also pays special attention to the design of human–computer interaction, provides an intuitive and easy-to-use user interface, and enables traffic managers to easily monitor and manage traffic conditions. This innovative intelligent vehicle violation detection system is expected to help the development of traffic management technology in the future.