Being increasingly present at the most diverse structure health monitoring (SHM) scenarios, many high-performance artificial intelligence techniques have been able to solve structural analysis problems. When it comes to image classification solutions, convolutional neural networks (CNNs) deliver the best results. This scenario encourages us to explore machine learning techniques, such as computer vision, and merge it with different technologies to achieve the best performance. This paper proposes a custom CNN architecture trained with slope erosion images that showed satisfactory results with an accuracy of 96.67%, enabling a precise and improved identification of instability indicators. These instabilities, when detected in advance, prevent disasters and enable proper maintenance to be carried out, given that its integrity directly affects structures built around and above it.