In our interconnected society, prioritizing the resilience and sustainability of road infrastructure has never been more critical, especially in light of growing environmental and climatic challenges. By harnessing data from various sources, we can proactively enhance our ability to detect road damage. This approach will enable us to make well-informed decisions for timely maintenance and implement effective mitigation strategies, ultimately leading to safer and more durable road systems. This paper presents a new method for detecting road potholes during low-light conditions, particularly at night when influenced by street and traffic lighting. We examined and assessed various advanced machine learning and computer vision models, placing a strong emphasis on deep learning algorithms such as YOLO, as well as the combination of Grad-CAM++ with feature pyramid networks for feature extraction. Our approach utilized innovative data augmentation techniques, which enhanced the diversity and robustness of the training dataset, ultimately leading to significant improvements in model performance. The study results reveal that the proposed YOLOv11+FPN+Grad-CAM model achieved a mean average precision (mAP) score of 0.72 for the 50–95 IoU thresholds, outperforming other tested models, including YOLOv8 Medium with a score of 0.611. The proposed model also demonstrated notable improvements in key metrics, with mAP50 and mAP75 values of 0.88 and 0.791, reflecting enhancements of 1.5% and 5.7%, respectively, compared to YOLOv11. These results highlight the model’s superior performance in detecting potholes under low-light conditions. By leveraging a specialized dataset for nighttime scenarios, the approach offers significant advancements in hazard detection, paving the way for more effective and timely driver alerts and ultimately contributing to improved road safety. This paper makes several key contributions, including implementing advanced data augmentation methods and a thorough comparative analysis of various YOLO-based models. Future plans involve developing a real-time driver warning application, introducing enhanced evaluation metrics, and demonstrating the model’s adaptability in diverse environmental conditions, such as snow and rain. The contributions significantly advance the field of road maintenance and safety by offering a robust and scalable solution for pothole detection, particularly in developing countries.