The increasing evaporative demand due to climate change will significantly affect the balance of carbon assimilation and water losses of plants worldwide. The development of crop varieties with improved water use efficiency (WUE) will be critical for adapting agricultural strategies under predicted future climate. This review aims to summarize the most important leaf morpho-physiological constraints of WUE in C3 plants and identify gaps in knowledge. From the carbon gain side of the WUE, the discussed parameters are mesophyll conductance, carboxylation efficiency and respiratory losses. The traits and parameters affecting the waterside of WUE balance discussed in this review are stomatal size and density, stomatal control and residual water losses (cuticular and bark conductance), nocturnal conductance and leaf hydraulic conductance. In addition, we discussed the impact of leaf anatomy and crown architecture on both the carbon gain and water loss components of WUE. There are multiple possible targets for future development in understanding sources of WUE variability in plants. We identified residual water losses and respiratory carbon losses as the greatest knowledge gaps of whole plant WUE assessments. Moreover, the impact of trichomes, leaf hydraulic conductance and canopy structure on plants’ WUE is still not well understood. The development of a multi-trait approach is urgently needed for a better understanding of WUE dynamics and optimization.