A soil removal device for the root-soil complex of Gentian imitating the percussion function of a woodpecker was designed to improve the soil removal efficiency of harvesting devices for rhizome-type traditional Chinese herbal medicines. Based on the physical parameters of roots and the root-soil complex of Gentian, the structure parameters of the striking arm and the actual profile of the cam are determined according to the physical parameters when the woodpecker knocks on the tree. The key parameters that affect the working performance of the soil removal device and their suitable value ranges have been identified through the impact test and analysis of the root-soil complex of Gentian. The mass of the striking hammer, the swing angle of the striking arm, and the rotation speed of the cam were taken as the experimental factors and the soil removal rate and the energy consumption per hammer percussion were taken as the experimental indicators. The ternary quadratic orthogonal regression combination experiment was carried out using Design-Expert. The regression model of the influence factors and evaluation indicators was established through the analysis of variance. The interaction effects of the influence factors on the indicators were analyzed using the response surface method. Using multiobjective optimization method, the optimal parameter combination was obtained as that of the mass of the striking hammer of 0.9 kg, the swing angle of the striking arm of 47°, and the rotation speed of the cam of 100 r/min, then the soil removal rate was the maximum and the energy consumption of single-hammer knocking was the minimum, with the values of 89.12% and 31.21 J, respectively. This study can provide a reference for the design and optimization of soil removal devices for rhizome-type traditional Chinese herbal medicines.