In the rapid development of urbanization, the sustained and healthy development of transportation infrastructure has become a widely discussed topic. The inspection and maintenance of asphalt pavements not only concern road safety and efficiency but also directly impact the rational allocation of resources and environmental sustainability. To address the challenges of modern transportation infrastructure management, this study innovatively proposes a hybrid learning model that integrates deep convolutional neural networks (DCNNs) and support vector machines (SVMs). Specifically, the model initially employs a ShuffleNet architecture to autonomously extract abstract features from various defect categories. Subsequently, the Maximum Relevance Minimum Redundancy (MRMR) method is utilized to select the top 25% of features with the highest relevance and minimal redundancy. After that, SVMs equipped with diverse kernel functions are deployed to perform training and prediction based on the selected features. The experimental results reveal that the model attains a high classification accuracy of 94.62% on a self-constructed asphalt pavement image dataset. This technology not only significantly improves the accuracy and efficiency of pavement inspection but also effectively reduces traffic congestion and incremental carbon emissions caused by pavement distress, thereby alleviating environmental burdens. It is of great significance for enhancing pavement maintenance efficiency, conserving resource consumption, mitigating environmental pollution, and promoting sustainable socio-economic development.