Anomaly object detection is the core technology in the application for X‐ray images. However, the accuracy of current X‐ray anomaly object detection method still needs to be improved. In this paper, an effective anomaly object detection network is proposed to improve the detection accuracy of anomaly object for X‐ray images. Firstly, learnable Gabor convolution layer, deformable convolution, and spatial attention mechanism are introduced to enhance the representative ability of features in ResNeXt. Then, dense local regression is applied to predict the offset of multiple dense boxes in region proposal to locate the object accurately. At last, bigger discriminative RoI pooling is proposed to classify the candidate boxes to improve the accuracy of object classification. Experimental results on the SIXray and OPIXray datasets show that compared with the state‐of‐the‐art methods, the proposed EAOD‐Net can achieve the competitive detection performance.