Identifying an object of interest in thermal images plays a vital role in several military and civilian applications. The deep learning approach has shown its superiority in object detection in various RGB datasets. However, regarding to thermal images, their low resolution and shortage of detail properties impose a huge challenge that hinders the accuracy. In this paper, we propose an improved version of YOLOv5 model to tackle this problem. Convolution Block Attention Module (CBAM) is integrated into traditional YOLOv5 for better representation of objects by focusing on important features and neglecting unnecessary ones. The Selective Kernel Network(SENet) is added to maximize the shallow features usage. Furthermore, the multiscale detection mechanism is utilized to improve small object detection accuracy. We train our model on the mixed visible-thermal images collected from LSOTB-TIR, LLVIP, and COCO datasets. We evaluate the performance of our method on 8 classes of objects: person, bicycle, airplane, helicopter, car, motorbike, boat, and tank. Experiment results show that our approach can achieve mAP up to 90.2%, which outperforms the original YOLOv5 and other popular methods.