There is a great demand for automatic detection and classification of blood cells (BCs) in clinical medical diagnoses. Traditional methods, such as hematology analyzer and manual counting were laborious, time intensive, and limited by analysts' professional experience and knowledge. In this paper, the one-stage network based upon improved YOLOv5s is provided to detect BCs. First, the Transformer and bidirectional feature pyramid network (BiFPN) are introduced into the backbone network and neck network for refining the adaptive features, respectively. Second, Convolutional Block Attention Module (CBAM) is added to neck network outputs to strengthen the key features in space and channel. In addition, an Efficient Intersection over Union (EIoU) was introduced to improve model accuracy regarding localization and performance. The improvements are embedded into the YOLOv5s model and termed YOLOv5s-TRBC. The experiments on the blood cell dataset (BCCD) show that in the three types of BCs detections, the mean average precision (mAP) of the method proposed reached 93.5%. Furthermore, comparative experiments demonstrate that the model could perform favorably against the counterparts with respect to mAP rate, and the model's Giga Floating-point Operations Per Second (GFLOPs) is reduced to 1/6 of YOLOv5, which provides a potential solution for future computer-aid diagnostic systems.