The transportation infrastructure sector significantly impacts a country’s gross domestic product (GDP), particularly in developing nations striving to manage and maintain road networks as valuable assets. While asset generation is integral, the more intricate challenge lies in effective maintenance. Pavement monitoring, a crucial component of pavement maintenance and management systems (PMMS), evaluates defect severity, road maintenance prioritization, and maintenance types. To enhance road health monitoring, the present study introduces a hybrid machine learning (ML) method, integrating support vector machine (SVM) and convolutional neural network (CNN). The proposed semi-automated detection system aims to reduce human supervision in traditional surveys, thereby cutting down the cost of pavement distress maintenance The research utilizes data collected by the authors from Ahmedabad city, Gujarat, following Indian road congress (IRC) guidelines for defect selection. Training involves 1,000 images for each crack type, with testing on 100 images. Results indicate that the SVM-CNN model achieves 87% accuracy in training and 91% accuracy in testing for road defect classification, showcasing its efficiency in pavement maintenance and management. The system presents the potential to significantly enhance the efficiency of road maintenance processes, making it a valuable asset for developing nations striving for a more streamlined approach to road network preservation.