The Nanyang Sag in the Nanxiang Basin, eastern China, has a large petroleum resource, although the genetic potential of source rocks and oil–rock correlation in the Paleogene Hetaoyuan Formation is unclear. Organic geochemistry and molecular geochemistry analyses of mudstone and oil samples were performed to evaluate source rock potential, classify oil groups, and establish accurate oil–source rock correlation. The results suggest that the second member (Eh2) and the third member (Eh3) of the Hetaoyuan Formation are good source rocks, contain abundant organic matter and oil‐prone kerogen, and are thermally mature. Oil samples from Eh2 and Eh3 were separated into 3 groups derived from different source rocks. Group I oils are mainly distributed in the Eh2 and Eh3 members around the Dongzhuang (Dongz) structure belt and are characterized by a low saturated hydrocarbon content (SAT, 39.07–49.54%), high total resin (RE) and asphaltene (ASP) content (17.22–22.80%), low gammacerane index (gammacerane/C3017α(H)‐hopane < 0.60), and low Pr/Ph (<0.7). The organic matter is in an early mature stage and is mainly sourced from algae and land plants. The oils of this group are derived from the Eh2 and Eh3 source rocks around the Dongz secondary sag. Group II oils occur in the Eh2 around the Weigang–Beimazhuang (Weig–Beimz) structure belt and north of the Zhangdian–Heilongmiao (Zhangd–Heilm) structure belt and have intermediate SAT, RE, and ASP contents, a low gammacerane index (<0.47), and high Pr/Ph (0.72–1.25). The organic matter includes mainly algae and land plants. The source rocks should be the Eh2 and Eh3 in the Dongz and Niusanmen (Niusm) secondary sags. Group III oils are located in the Eh3 just north of the Weig–Beimz and Zhangd–Heilm structure belts, with a high SAT (55.62–78.42%), low RE and ASP proportion (3.60–6.16%), high gammacerane index (>0.88), and low Pr/Ph (<0.72). This organic matter is composed of mostly land plants. The oils in this group are mainly derived from the Eh3 near Baiqiu (Baiq).