A smart city is the future goal for humanity to have cleaner and better amenities. When creating a smart city, smart subsurface infrastructure is a crucial component to consider. Monitoring the drainage system is critical to keeping the city clean and healthy. Because human monitoring is ineffective, drainage problems are handled slowly and take longer to resolve. The technology is designed to plot the co-ordinates of the manholes using GPS.The proposed approach is low-cost and requires little maintenance. The GPS will refresh the information every 5 minutes. When crossing a manhole, the driver is alerted that a manhole is ahead, allowing them to avoid it. The system employs a machine learning algorithm based on Q-Learning to calculate each state. If any of those parameters changes abruptly, the system connects to the admin office and notifies the situation via message wherever it has been posted. This study would be tremendously beneficial to society in terms of manhole maintenance. Implement a Reinforcement Learning algorithm, such as Q-Learning, that is based on reward and policy. A test data set will be used to assess the performance of the Q learning model.