The vehicle detection method plays an important role in the driver assistance system. Therefore, it is very important to improve the real-time performance of the detection algorithm. Nowadays, the most popular method is the scanning method based on sliding window search, which detects the vehicle from the image to be detected. However, the existing sliding window detection algorithm has many drawbacks, such as large calculation amount and poor real-time performance, and it is impossible to detect the target vehicle in real time during the motion process. Therefore, this paper proposes an improved hierarchical sliding window detection algorithm to detect moving vehicles in real time. By extracting the region of interest, the region of interest is layered, the maximum and minimum values of the detection window in each layer are set, the flashing frame generated by the layering is eliminated by the delay processing method, and a method suitable for the motion is obtained: the real-time detection algorithm of the vehicle, that is, the hierarchical sliding window detection algorithm. The experiments show that the more layers are divided, the more time is needed, and when the number of detection layers is greater than 7, the time change rate increases significantly. As the number of layers decreases, the detection accuracy rate also decreases, resulting in the phenomenon of a false positive. Therefore, it is determined to meet the requirements of real time and accuracy when the image is divided into 7 layers. It can be seen from the experiment that when the images to be detected are divided into 7 layers and the maximum and minimum values of detection windows are 30 × 30 and 250 × 250, respectively, the number of sub-windows generated is one thirty-seventh of the original sliding window detection algorithm, and the execution time is only one-third of the original sliding window detection algorithm. This shows that the hierarchical sliding window detection algorithm has better real-time performance than the original sliding window detection algorithm.