Aiming to quickly and accurately detect abnormal behaviors of workers in central control rooms, such as playing mobile phone and sleeping, an abnormal behavior detection algorithm based on improved YOLOv5 is proposed. The technique uses SRGAN to reconstruct the input image to improve the resolution and enhance the detailed information. Then, the MnasNet is introduced to replace the backbone feature extraction network of the original YOLOv5, which could achieve the lightweight of the model. Moreover, the detection accuracy of the whole network is enhanced by adding the ECA-Net attention mechanism into the feature fusion network structure of YOLOv5 and modifying the loss function as EIOU. The experimental results in the custom dataset show that compared with the original YOLOv5 algorithm, the algorithm proposed in this paper improves thedetection speed to 75.50 frames/s under the condition of high detection accuracy, which meets the requirements of real-time detection. Meanwhile, compared with other mainstream behavior detection algorithms, this algorithm also shows better detection performance.