High-resolution images provided by synthetic aperture radar (SAR) play an increasingly important role in the field of ship detection. Numerous algorithms have been so far proposed and relative competitive results have been achieved in detecting different targets. However, ship detection using SAR images is still challenging because these images are still affected by different degrees of noise while inshore ships are affected by shore image contrasts. To solve these problems, this paper introduces a ship detection method called N-YOLO, which based on You Only Look Once (YOLO). The N-YOLO includes a noise level classifier (NLC), a SAR target potential area extraction module (STPAE) and a YOLOv5-based detection module. First, NLC derives and classifies the noise level of SAR images. Secondly, the STPAE module is composed by a CA-CFAR and expansion operation, which is used to extract the complete region of potential targets. Thirdly, the YOLOv5-based detection module combines the potential target area with the original image to get a new image. To evaluate the effectiveness of the N-YOLO, experiments are conducted using a reference GaoFen-3 dataset. The detection results show that competitive performance has been achieved by N-YOLO in comparison with several CNN-based algorithms.