The development of ecological restoration projects is unsatisfactory, and soil erosion is still a problem in ecologically restored areas. Traditional soil erosion studies are mostly based on satellite remote sensing data and traditional soil erosion models, which cannot accurately characterize the soil erosion conditions in ecological restoration areas (mainly plantation forests). This paper uses high-resolution unmanned aerial vehicle (UAV) images as the base data, which could improve the accuracy of the study. Considering that traditional soil erosion models cannot accurately express the complex relationships between erosion factors, this paper applies convolutional neural network (CNN) models to identify the soil erosion intensity in ecological restoration areas, which can solve the problem of nonlinear mapping of soil erosion. In this study area, compared with the traditional method, the accuracy of soil erosion identification by applying the CNN model improved by 25.57%, which is better than baseline methods. In addition, based on research results, this paper analyses the relationship between land use type, vegetation cover, and slope and soil erosion. This study makes five recommendations for the prevention and control of soil erosion in the ecological restoration area, which provides a scientific basis and decision reference for subsequent ecological restoration decisions.