Congested regions in videos put forward higher requirements for target detection algorithms, and the key detection of congested regions provides optimization directions for improving the accuracy of detection algorithms. In order to make the target detection algorithm pay more attention to the congested area, an automatic selection method of a traffic congestion area based on surveillance videos is proposed. Firstly, the image is segmented with superpixels, and a superpixel boundary map is extracted. Then, the mean filtering method is used to process the superpixel boundary map, and a fixed threshold is used to filter pixels with high texture complexity. Finally, a maximin method is used to extract the traffic congestion area. Monitoring data of night and rainy days were collected to expand the UA-DETRAC data set, and experiments were carried out on the extended data set. The results show that the proposed method can realize automatic setting of the congestion area under various weather conditions, such as full light, night and rainy days.