The development of Traffic Sign Recognition (TSR) has become increasingly important for enhancing the safety and convenience of assisted driving. To achieve high accuracy, faster inference speed, and a lightweight model, an improved lightweight Traffic Sign Recognition network, termed YOLOv8-ALWP, has been proposed. This network incorporates Adaptive Downsampling (ADown) to replace the original convolution module in YOLOv8. By employing multiple pooling and convolution operations, it reduces the spatial resolution to extract additional feature information. To accommodate the varying scale characteristics of different traffic signs, Large Separable Kernel Attention (LSKA) is introduced to enhance Spatial Pyramid Pooling-Fast (SPPF). Furthermore, the Complete Intersection over Union (CIoU) loss has been improved, and a new Wise-Focaler-EIoU Loss has been proposed to accelerate model convergence and enhance generalization capabilities. Finally, Layer-Adaptive Sparsity for Magnitude-Based Pruning (LAMP) is employed to reduce the model's parameters, decrease computational complexity, and improve inference speed. Experiments were conducted using the TT100K, Roadsign, CCTSDB, and GTSDB datasets. In the TT100K dataset, compared to the baseline model, the improved algorithm significantly reduced parameters by 64.67%, FLOPs by 44.44%, and increased mAP by 1.7%, precision by 5.5%, and FPS from 70.3 to 81.7, respectively. Under four specific conditions, the improved algorithm effectively addressed the shortcomings of the baseline model, such as missed detections and reduced accuracy. These experimental results indicate that the YOLOv8-ALWP algorithm achieves model lightweighting while enhancing detection accuracy.