Selenium (Se) is a micronutrient that has attracted significant attention, because the threshold for human health is low. During soil surveys in China, large areas of low-Se soil were found, and this condition may increase the probability of people suffering from Se deficiency. A multi-purpose regional geochemical survey conducted in the Lou Shao basin of Hunan Province found abundant Se-rich soils in Lianyuan City. However, as the primary grain-producing area in Hunan Province, the key factors affecting the spatial distribution of soil Se in the cultivated land of Lianyuan City remain to be elucidated. Therefore, based on the data of 5516 topsoil samples (0–20 cm) of cultivated land in Lianyuan City, we used geostatistics, correlation analysis, and a Geodetector to explore the effects of geological conditions (strata), soil types, soil properties, and topography on the distribution of Se in soil. The results showed that (1) in comparison to cultivated land in the Chinese mainland, Japan, Belgium, and Sweden, the cultivated land in Lianyuan City exhibits higher Se contents, with Se-sufficient and Se-rich areas accounting for 9.74% and 88.96% of the total area, respectively; (2) the distribution of high-Se soil was consistent with that in the Longtan Formation, Dalong Formation, and Daye Formation; (3) organic matter (OM) showed a positive correlation with Se, while both the elevation and slope were negatively correlated with Se; (4) stratum had the most significant effect on the spatial variation in soil Se, followed by OM. Lianyuan City is a typical Se-rich area, and the high level of Se in soil reduces the risk of local residents suffering with diseases caused by Se deficiency. The synergistic effect of stratum and OM is the key factor influencing Se enrichment in soils. Moreover, low-lying flat areas are more conducive to the accumulation of Se. This study will help farmers to identify suitable Se-rich cultivation areas in order to increase the Se content in crops, thereby providing a valuable basis for improvements in human health and the optimization of agricultural strategies.