In recent years, construction accidents have occurred frequently. The safety guarantee measures for construction personnel are thrown into sharp focus. Wearing helmets is one of the most important requirements to protect the safety of construction personnel, and the detection of wearing safety helmets has become necessary. For the problems of the existing helmet wearing detection algorithm such as too many parameters, substantial detection interferences, and low detection accuracy, in this paper a helmet wearing detection model YOLO-M is proposed. Firstly, MobileNetv3 is adopted as the backbone network of YOLOv5s for the feature extraction, which can reduce the number of model parameters and model size. Secondly, a residual edge is introduced in the feature fusion. The original feature map information is fused during feature fusion, and the detection ability of small targets is enhanced. At last, by changing the connection between CAM and SAM, a new attention module BiCAM is designed. The comparison experiments show that the detection accuracy of YOLO-M is 2.22% higher than YOLOv5s, and the model parameter quantity is reduced to 3/4 of YOLOv5s. Under the same detection conditions, the detection speed of YOLO-M is better than the other models, which meets the accuracy requirements of helmet detection in the construction scene.INDEX TERMS Attention mechanism, feature fusion, safety helmet, YOLOv5s model.