Intelligent medical robots can effectively help doctors carry out a series of medical diagnoses and auxiliary treatments and alleviate the current shortage of social personnel. Therefore, this paper investigates how to use deep reinforcement learning to solve dynamic medical auscultation tasks. We propose a constant force-tracking control method for dynamic environments and a modeling method that satisfies physical characteristics to simulate the dynamic breathing process and design an optimal reward function for the task of achieving efficient learning of the control strategy. We have carried out a large number of simulation experiments, and the error between the tracking of normal force and expected force is basically within ±0.5 N. The control strategy is tested in a real environment. The preliminary results show that the control strategy performs well in the constant force-tracking of medical auscultation tasks. The contact force is always within a safe and stable range, and the average contact force is about 5.2 N.