Landslides are geological disasters that can cause serious severe damage to properties and lead to the loss of human lives. The application of deep learning technology to optical remote sensing images can help in the detection of landslide areas. Traditional landslide detection models usually have complex structural designs to ensure accuracy. However, this complexity leads to slow detection, and these models often do not satisfy the rapid response required for the emergency monitoring of landslides. Therefore, we designed a lightweight landslide target detection network based on a CenterNet and a ResNet50 network. We replaced the BottleNeck in the backbone network of ResNet50 with a Ghost-BottleNeck structure to reduce the number of parameters in the model. We also introduced an attention mechanism module based on channel attention and spatial attention between the adjacent GhostModule modules to rich the landslide features. We introduced a lightweight multiscale fusion method in the decoding process that presented a cross-layer sampling operation for the encoding process based on Feature Pyramid Network. To down-sample from a low resolution to a high resolution and up-sample from a high resolution to a low resolution, thus skipping the medium-resolution levels in the path. We added the feature maps obtained in the previous step to the feature fusion. The Conv module that adjusts the number of channels in the multiscale feature fusion operation was replaced with the GhostModule to achieve lightweight capability. At the end of the network, we introduced a state-of-the-art Yolov5x as a teacher network for feature-based knowledge distillation to further improve the accuracy of our student network. We used challenging datasets including multiple targets and multiscale landslides in the western mountains of Sichuan, China (e.g., Danba, Jiuzhaigou, Wenchuan, and Maoxian) to evaluate the proposed lightweight landslide detection network. The experimental results show that our model satisfied landslide emergency requirements in terms of both accuracy and speed; the parameter size of the proposed lightweight model is 18.7 MB, namely, 14.6% of the size of the original CenterNet containing the ResNet50 network. The single image detection time is 52 ms—twice as fast as the original model. The detection accuracy is 76.25%, namely, 12% higher than that of the original model.