Increasing crop yields can provide food, animal feed, bioenergy feedstocks and biomaterials to meet increasing global demand; however, the methods used to increase yield can negatively affect sustainability. For example, application of excess fertilizer can generate and maintain high yields but also increases input costs and contributes to environmental damage through eutrophication, soil acidification and air pollution. Improving crop nutrient efficiency can improve agricultural sustainability by increasing yield while decreasing input costs and harmful environmental effects. Here, we review the mechanisms of nutrient efficiency (primarily for nitrogen, phosphorus, potassium and iron) and breeding strategies for improving this trait, along with the role of regulation of gene expression in enhancing crop nutrient efficiency to increase yields. We focus on the importance of root system architecture to improve nutrient acquisition efficiency, as well as the contributions of mineral translocation, remobilization and metabolic efficiency to nutrient utilization efficiency.