Selenium (Se) is an essential element for humans and animals and its deficiency in the diet is a global problem. Crop plants are the main source of Se for consumers. Therefore, there is much interest in understanding the factors that govern the accumulation and distribution of Se in the tissues of crop plants and the mechanisms of interaction of Se absorption and accumulation with other elements, especially with a view toward optimizing Se biofortification. An ideal crop for human consumption is rich in essential nutrient elements such as Se, while showing reduced accumulation of toxic elements in its edible parts. This review focuses on (a) summarizing the nutritional functions of Se and the current understanding of Se uptake by plant roots, translocation of Se from roots to shoots, and accumulation of Se in grains; and (b) discussing the influence of nitrogen (N), phosphorus (P), and sulfur (S) on the biofortification of Se. In addition, we discuss interactions of Se with major toxicant metals (Hg, As, and Cd) frequently present in soil. We highlight key challenges in the quest to improve Se biofortification, with a focus on both agronomic practice and human health.