The urban drainage system is an important part of the urban water cycle. However, with the aging of drainage pipelines and other external reasons, damages such as cracks, corrosion, and deformation of underground pipelines can cause serious consequences such as urban waterlogging and road collapse. At present, the detection of underground drainage pipelines mostly focuses on the qualitative identification of pipeline damage, and it is impossible to quantitatively analyze pipeline damage. Therefore, a method to quantify the damage volume of concrete pipes that combines surface segmentation and reconstruction is proposed. An RGB-D sensor is used to collect the damage information of the drainage pipeline, and the collected depth frame is registered to generate the pipeline’s surface point cloud. Voxel sampling and Gaussian filtering are used to improve data processing efficiency and reduce noise, respectively, and the RANSAC algorithm is used to remove the pipeline’s surface information. The ball-pivoting algorithm is used to reconstruct the surface of the segmented damage data and pipe’s surface information, and finally to obtain the damage volume. In order to evaluate, we conducted our research on real-world materials. The measurement results show that the method proposed in this paper measures an average relative error of 7.17% for the external damage volume of concrete pipes and an average relative error of 5.22% for the internal damage measurements of concrete pipes.