Nanmu (Phoebe zhennan) has a unique fragrance and is a high-quality tree species for forest conservation. The types and contents of volatile compounds in different tissues of nanmu wood are different, and the study of its volatile metabolites can help us to understand the source of its fragrance and functions. In order to explore the metabolites related to the wood fragrance of nanmu and to find out the unique volatile substances in the heartwood, gas chromatography–mass spectrometry (GC-MS) was performed to analyze the non-targeted metabolomics in five radial tissues from the sapwood to the heartwood of nanmu. A total of 53 volatile metabolites belonging to 11 classes were detected in all tissues, including terpenes, aromatic hydrocarbons, organoheterocyclics, phenols, esters, organic acids, alcohols, alkaloids, alkane, indoles derivatives, and others. And most of the volatile metabolites were identified for the first time in nanmu wood. Among them, terpenes and aromatic hydrocarbons were the main volatile components. In addition, 22 differential metabolites were screened from HW and SW, HW, and TZ via metabolomic analysis. Among these DAMs, three volatile metabolites (cadinene, a sesquiterpenoid; p-cymene, a monoterpenoid; 1,3,5-triisopropylbenzene, an aromatic hydrocarbon) contributed heavily to the characteristic fragrance of the heartwood. Additionally, the expression of transcripts showed that the unigenes in the terpenoid biosynthesis pathway were especially up-regulated in the SW. Therefore, we speculated that fragrance-related metabolites were synthesized in SW and then deposited in heartwood during sapwood transformed to heartwood. The expression levels of transcription factors (e.g., WRKY, C2H2, NAC) acted as the major regulatory factors in the synthesis of terpenoid. The results lay the foundations for further studies on the formation mechanism of fragrance components in nanmu wood and also provide a reference for the further development and utilization of nanmu wood.