Potentially toxic element (PTE) pollution in farmland soils and crops is a serious cause of concern in China. To analyze the bioaccumulation characteristics of chromium (Cr), zinc (Zn), copper (Cu), and nickel (Ni) in soil-rice systems, 911 pairs of top soil (0–0.2 m) and rice samples were collected from an industrial city in Southeast China. Multiple linear regression (MLR), support vector machines (SVM), random forest (RF), and Cubist were employed to construct models to predict the bioaccumulation coefficient (BAC) of PTEs in soil–rice systems and determine the potential dominators for PTE transfer from soil to rice grains. Cr, Cu, Zn, and Ni contents in soil of the survey region were higher than corresponding background contents in China. The mean Ni content of rice grains exceeded the national permissible limit, whereas the concentrations of Cr, Cu, and Zn were lower than their thresholds. The BAC of PTEs kept the sequence of Zn (0.219) > Cu (0.093) > Ni (0.032) > Cr (0.018). Of the four algorithms employed to estimate the bioaccumulation of Cr, Cu, Zn, and Ni in soil–rice systems, RF exhibited the best performance, with coefficient of determination (R2) ranging from 0.58 to 0.79 and root mean square error (RMSE) ranging from 0.03 to 0.04 mg kg−1. Total PTE concentration in soil, cation exchange capacity (CEC), and annual average precipitation were identified as top 3 dominators influencing PTE transfer from soil to rice grains. This study confirmed the feasibility and advantages of machine learning methods especially RF for estimating PTE accumulation in soil–rice systems, when compared with traditional statistical methods, such as MLR. Our study provides new tools for analyzing the transfer of PTEs from soil to rice, and can help decision-makers in developing more efficient policies for regulating PTE pollution in soil and crops, and reducing the corresponding health risks.