Accurately understanding the critical threshold of leaf magnesium (Mg) concentration is crucial for rapid diagnosis of crop Mg status; however, little information is available on critical Mg concentration for different physiological processes in dicots and monocots. Here, we investigated the sensitivity of biomass, chlorophyll (Chl) at different leaf positions/ages, leaf area (LA), and photosynthesis (Pn) to Mg deficiency between rice (Oryza sativa L.) and cucumber (Cucumis sativus L.). Plants were grown hydroponically under twelve Mg concentration gradients. Results showed reducing the external Mg supply to a certain level resulted in significant decline in biomass, Chl, LA, and Pn in both plants. A leaf Mg threshold of 0.97 mg g−1 DM (dry matter) for total biomass was found in rice, which was not identified in cucumber. Critical Mg thresholds for Chl a, b, and carotenoids (Car) showed a decreasing trend with leaf age, suggesting Chl in upper young leaves are more sensitive to Mg deficiency; however, visible Mg-deficiency symptoms were predominantly in mid-aged leaves with a higher rate of Mg remobilization, especially in cucumber. Leaf critical Mg concentrations for Chl a+b, Pn, and LA were 1.22, 1.05, and 1.00 mg g−1 DM in rice, respectively, which were lower than those of cucumber, 4.23, 4.09, and 3.55 mg g−1 DM, implying that cucumber was more susceptible to low Mg stress; Chl a+b was the most sensitive indicator of Mg deficiency. Overall, Chl a+b of upper young mature leaves can be used as an early diagnostic index of Mg nutrition in crops, especially Mg-insensitive crops.