Impact craters refer to the most salient features on the moon surface. They are of huge significance for analyzing the moon topography, selecting the lunar landing site and other lunar exploration missions, etc. However, existing methods of impact crater detection have been largely implemented on the optical image data, thereby causing them to be sensitive to the sunlight. Thus, these methods can easily achieve unsatisfactory detection results. In this study, an original two-stage small crater detection method is proposed, which is sufficiently effective in addressing the sunlight effects. At the first stage of the proposed method, a semantic segmentation is conducted to detect small impact craters by fully exploiting the elevation information in the digital elevation map (DEM) data. Subsequently, at the second stage, the detection accuracy is improved under the special post-processing. As opposed to other methods based on DEM images, the proposed method, respectively, increases the new crusher percentage, recall and crusher level F1 by 4.89%, 5.42% and 0.67%.