X‐ray fluorescence (XRF) analysis is exceedingly suitable for detecting heavy metal contents in soil. In order to do that, an accurate prediction model based on XRF analysis is necessary. But in practice, the XRF spectral data is susceptible to moisture content in soil, which may lead to inaccurate prediction results. Accordingly, a new prediction model based on Random Forest Regression (RFR) and improved Sparrow Search Algorithm (SSA) was proposed, which takes the variation of moisture content into consideration. At first, the XRF spectral data were obtained by experiment. Owing to the advantages of training speed and prediction ability, the RFR was employed to predict the heavy metal contents. In order to further improve the performance of RFR, the SSA was selected and improved with theory of good‐point set, which can determine optimum hyper‐parameters of RFR conveniently. It can be found by comparison that the proposed model outperforms other commonly used models.