Ship detection plays a vital role in monitoring and managing maritime safety. Most recently proposed learning-based object detection methods have achieved marked progress in detection accuracy, but the size of these models is too large to be applied to mobile devices with limited resources. Although some compact models have been presented in the previous study, they achieve unsatisfactory results in ship detection, especially under extreme weather conditions. To address these challenges, this article presents a lightweight convolutional neural network (CNN) called Light-SDNet to perform an end-to-end ship detection under different weather conditions. In the proposed model, we introduce the improved CA-Ghost, C3Ghost, and DepthWise Convolution (DWConv) into the You Only Look Once version 5 (YOLOv5) to reduce the number of model parameters, while remaining its powerful feature expression ability. We use parallel attention to highlight the features that contribute to the ship detection in the marine surveillance. To enhance the adaptability of the proposed model, a hybrid training strategy with generating synthetically-degraded images is proposed to augment the volume and diversity of the original datasets. The proposed strategy enables Light-SDNet to improve the ship detection results under severe weather conditions such as haze, rain, and low illumination. We compare Light-SDNet with other competitive approaches on a large-scaled ship dataset called SeaShips. We show that Light-SDNet achieves a better balance between the detection accuracy and the model complexity. The ship detection results on degraded marine images have proven the superior performance of the proposed model in terms of detection accuracy, robustness and efficiency.