Polyacrylonitrile‐based carbon fibers (CFs) have demonstrated unparalleled physical and mechanical properties and are considered a promising material for aerospace, transportation, medical devices, and other applications. However, the current understanding of the heterogeneous structure of carbon fiber is still vague, and the relationship between the heterogeneous structure and the mechanical properties of carbon fiber is also unclear. In this review, the literature on the radial heterogeneous structure of CFs is summarized from four points of view: morphological structure, physical properties and chemical structures, mechanical properties, and structural models by combining different characterization tools such as nanoindentation, nanoscale infrared spectroscopy, and synchrotron wide‐angle X‐ray diffraction. First, the size and orientation of graphite microcrystalline, the degree of graphitization, density distribution, modulus distribution, chemical functional group distribution, and other physical and chemical properties of carbon fiber cross‐sectional structures are discussed. Afterward, the relationship between the mechanical properties of carbon fiber and the heterogeneous structure is analyzed. Finally, the currently accepted models of the heterogeneous structure of carbon fibers are summarized and made some suggestions for research on the heterogeneous structure of carbon fibers.