Magnesium (Mg) deficiency, a widespread yet overlooked problem in agriculture, has been reported to retard plant growth and development, through affecting key metabolic pathways. However, the metabolic responses of plant to Mg deficiency is still not fully understood. Here we report a metabolomic study to evaluate the metabolic responses to Mg deficiency in soybean leaves and roots. Hydroponic grown soybean were exposed to Mg starvation for 4 and 8 days, respectively. Metabolic changes in the first mature trifoliolate leaves and roots were quantified by conducting GC-TOF-MS based metabolomic analysis. Principal component analysis (PCA) showed that Mg deficient plants became distinguishable from controls at 4 days after stress (DAS) at metabolic level, and were clearly discriminated at 8 DAS. Mg deficiency could cause large metabolite alterations on carbon and nitrogen metabolism. At 8 DAS, carbon allocation from shoot to root is decreased by Mg deficiency. Remarkably, most amino acids (such as phenylalanine, asparagine, leucine, isoleucine, glycine, glutamine, and serine) showed pronounced accumulation in the leaves, while most organic acids (including pyruvic acid, citric acid, 2-keto-glutaric acid, succinic acid, fumaric acid, and malic acid) were significantly decreased in the roots. Our study shows that the carbon and nitrogen metabolic responses are distinct in leaves and roots under Mg deficiency.