Potholes have been and still are a huge problem for every walk of life. There are many deaths and accidents reported daily due to that very problem. For that reason, pothole recognition comes into the picture. To maintain and preserve a road, it is vital to detect potholes. It also helps in the prevention of accidents. Roads play an important part in day-to-day transportation for every person around the world. But the quality of roads decreases drastically due to the way of usage and aging. The existing methods take much time and manpower to repair the damaged areas. The entire process is slowing down just because an expert team is checking whether there is a pothole at the reported location or not. So, if we automate the process of detection of potholes from a set of images reported from a particular location and appropriately alerting the authorities with the amount of damage, the process speeds up exponentially. We must solve the major problem of pothole recognition by using machine learning algorithms. This paper will discuss a convolution neural network-based and a transfer learning-based solution for pothole recognition.