Terraces, which are typical artificial landforms found around world, are of great importance for agricultural production and soil and water conservation. However, due to the lack of maintenance, terrace damages often occur and affect the local flow process, which will influence soil erosion. Automatic high-accuracy mapping of terrace damages is the basis of monitoring and related studies. Researchers have achieved artificial terrace damage mapping mainly via manual field investigation, but an automatic method is still lacking. In this study, given the success of high-resolution unmanned aerial vehicle (UAV) photogrammetry and object-based image analysis (OBIA) for image processing tasks, an integrated framework based on OBIA and UAV photogrammetry is proposed for terrace damage mapping. The Pujiawa terrace in the Loess Plateau of China was selected as the study area. Firstly, the segmentation process was optimised by considering the spectral features and the terrains and corresponding textures obtained from high-resolution images and digital surface models. The feature selection was implemented via correlation analysis, and the optimised segmentation parameter was achieved using the estimation of scale parameter algorithm. Then, a supervised k-nearest neighbourhood classifier was used to identify the terrace damages in the segmented objects, and additional geometric features at the object level were considered for classification. The comparison with the ground truth, as delineated by the image and field survey, showed that proposed classification can be adequately performed. The F-measures of extraction on three terrace damages were 92.07% (terrace sinkhole), 81.95% (ridge sinkhole), and 85.17% (collapse), and the Kappa coefficient was 85.34%. Finally, the potential application and spatial distribution of the terrace damages in this study were determined. We believe that this work can provide a credible framework for mapping terrace damages in the Loess Plateau of China.