Maritime ship detection plays a crucial role in smart ships and intelligent transportation systems. However, adverse maritime weather conditions, such as rain streak and fog, can significantly impair the performance of visual systems for maritime traffic. These factors constrain the performance of traffic monitoring systems and ship-detection algorithms for autonomous ship navigation, affecting maritime safety. The paper proposes an approach to resolve the problem by visually removing rain streaks and fog from images, achieving an integrated framework for accurate ship detection. Firstly, the paper employs an attention generation network within an adversarial neural network to focus on the distorted regions of the degraded images. The paper also utilizes a contextual encoder to infer contextual information within the distorted regions, enhancing the credibility of image restoration. Secondly, a weighted bidirectional feature pyramid network (BiFPN) is introduced to achieve rapid multi-scale feature fusion, enhancing the accuracy of maritime ship detection. The proposed GYB framework was validated using the SeaShip dataset. The experimental results show that the proposed framework achieves an average accuracy of 96.3%, a recall of 95.35%, and a harmonic mean of 95.85% in detecting maritime traffic ships under rain-streak and foggy-weather conditions. Moreover, the framework outperforms state-of-the-art ship detection methods in such challenging weather scenarios.