The characteristics of pulmonary tuberculosis are complex, and the cost of manual screening is high. The detection model based on convolutional neural network is an essential method for assisted diagnosis with artificial intelligence. However, it also has the disadvantages of complex structure and a large number of parameters, and the detection accuracy needs to be further improved. Therefore, an improved lightweight YOLOv4 pulmonary tuberculosis detection model named MIP-MY is proposed. Firstly, over 300 actual cases are selected to make a common dataset by professional physicians, which is used to evaluate the performance of the model. Subsequently, by introducing the inverted residual channel attention and the pyramid pooling module, a new structure of MIP is created and used as the backbone extractor of MIP-MY, which could further decrease the number of parameters and fuse context information. Then the multiple receptive field module is added after the three effective feature layers of the backbone extractor, which effectively enhances the information extraction ability of the deep feature layer and reduces the miss detection rate of small pulmonary tuberculosis lesions. Finally, the pulmonary tuberculosis detection model MIP-MY with lightweight and multiple receptive field characteristics is constructed by combining each improved modules with multiscale structure. Compared to the original YOLOv4, the model parameters of MIP-MY is reduced by 47%, while the mAP value is raised to 95.32% and the miss detection rate is decreased to 6%. It is verified that the model can effectively assist radiologists in the diagnosis of pulmonary tuberculosis.