Understanding the genetic structure and diversity of marine fish is crucial for a sustainable management program. We examined the genetic diversity and historical demographics of the yellowfin snapper, Lutjanus xanthopinnis Iwatsuki, Tanaka et Allen, 2015, in the coastal waters of east Peninsular Malaysia which is bordered by the southern part of the South China Sea using the mitochondrial genes (mtDNA) D-loop and Cytochrome b (Cyt-b). A total of 99 (D-loop) and 78 (Cyt-b) specimens of L. xanthopinnis were successfully sequenced from six locations within the range of species distribution along the Malaysian South China Sea. In the presently reported study, the lack of genetic differentiation among populations can be attributed to historical demographic events, eggs and planktonic larvae’ ability to disperse, spawning patterns, and the absence of physical barriers in the geographical landscape. Maximum likelihood gene trees demonstrated that the populations under study had limited structuring and formed a panmictic population that lacks support for internal clades. The AMOVA (Analysis of Molecular Variance) and population pairwise ФST values indicated high genetic exchange between the study areas. A high level of haplotype diversity (D-loop: 0.948–1.000; Cyt-b: 0.542–0.928), low nucleotide diversity (D-loop: 0.0095–0.0159; Cyt-b: 0.0022–0.0049) and starlike haplotype network indicates a recent expansion of L. xanthopinnis populations in Malaysian South China Sea. However, neutrality and goodness of fit tests revealed non-significant values. Furthermore, the BSP (Bayesian skyline plot) analysis estimated population expansion events during the late Pleistocene. During this epoch, the fluctuation in sea level may have led to an increase in the abundance of resources and favorable habitats for the yellowfin snapper. The presently reported findings could initiate efficient management strategies for L. xanthopinnis along the coastal areas of the Malaysian South China Sea and other nearby nations that share the same waterways.