Computer-aided diagnosis (CAD) has nearly fifty years of history and has assisted many clinicians in the diagnosis. With the development of technology, recently, researches use the deep learning method to get high accuracy results in the CAD system. With CAD, the computer output can be used as a second choice for radiologists and contribute to doctors doing the final right decisions. Chest abnormality detection is a classic detection and classification problem; researchers need to classify common thoracic lung diseases and localize critical findings. For the detection problem, there are two deep learning methods: one-stage method and two-stage method. In our paper, we introduce and analyze some representative model, such as RCNN, SSD, and YOLO series. In order to better solve the problem of chest abnormality detection, we proposed a new model based on YOLOv5 and ResNet50. YOLOv5 is the latest YOLO series, which is more flexible than the one-stage detection algorithms before. The function of YOLOv5 in our paper is to localize the abnormality region. On the other hand, we use ResNet, avoiding gradient explosion problems in deep learning for classification. And we filter the result we got from YOLOv5 and ResNet. If ResNet recognizes that the image is not abnormal, the YOLOv5 detection result is discarded. The dataset is collected via VinBigData’s web-based platform, VinLab. We train our model on the dataset using Pytorch frame and use the mAP, precision, and F1-score as the metrics to evaluate our model’s performance. In the progress of experiments, our method achieves superior performance over the other classical approaches on the same dataset. The experiments show that YOLOv5’s mAP is 0.010, 0.020, 0.023 higher than those of YOLOv5, Fast RCNN, and EfficientDet. In addition, in the dimension of precision, our model also performs better than other models. The precision of our model is 0.512, which is 0.018, 0.027, 0.033 higher than YOLOv5, Fast RCNN, and EfficientDet.