Due to the challenges of small detection targets, dense target distribution, and complex backgrounds in aerial images, existing object detection algorithms perform poorly in aerial image detection tasks. To address these issues, this paper proposes an improved algorithm called YOLOv5s-DSD based on YOLOv5s. Specifically, the SPDA-C3 structure is proposed and used to reduce information loss while focusing on useful features, effectively tackling the challenges of small detection targets and complex backgrounds. The novel decoupled head structure, Res-DHead, is introduced, along with an additional small object detection head, further improving the network’s performance in detecting small objects. The original NMS is replaced by Soft-NMS-CIOU to address the issue of neighboring box suppression caused by dense object distribution. Finally, extensive ablation experiments and comparative tests are conducted on the VisDrone2019 dataset, and the results demonstrate that YOLOv5s-DSD outperforms current state-of-the-art object detection models in aerial image detection tasks. The proposed improved algorithm achieves a significant improvement compared with the original algorithm, with an increase of 17.4% in mAP@0.5 and 16.4% in mAP@0.5:0.95, validating the superiority of the proposed improvements.