Detecting dangerous goods in security images is a challenging task. To overcome the challenges of localization difficulty and directional feature loss of contraband in X-ray images, our proposed solution, R3Det, employs the Convolutional Block Attention Module (CBAM). By integrating ResNeSt into the original detector, our detector includes a soft attention mechanism to redistribute weights among feature channels. This enhances the network’s ability to extract important features and facilitates extraction of target objects features under complex backgrounds. Subsequently, we introduced the spatial and channel attention mechanism during the connection between the backbone and the Feature Pyramid Network (FPN), enabling the model to focus on significant features while ignoring complex background information, then the following Feature Refinement Module to achieve feature alignment in a pixel-by-pixel manner. Our approach successfully achieved rotating target detection in the background of complex X-ray images. Through end-to-end training, our proposed method achieves a 2.6% improvement over the original detector, with a mean Average Precision (mAP) of 86.7%. Notably, our approach showed remarkable results in detecting sensors, pressure, and firetrackers. Now, we have deployed our proposed method on actual security machines for hazardous material detection tasks.