Pogostemon Desf., the largest genus of the tribe Pogostemoneae (Lamiaceae), consists of ca. 80 species distributed mainly from South and Southeast Asia to China. The genus contains many patchouli plants, which are of great economic importance but taxonomically difficult. Therefore, it is necessary to characterize more chloroplast (cp) genomes for infrageneric phylogeny analyses and species identification of Pogostemon, especially for patchouli plants. In this study, we newly generated four cp genomes for three patchouli plants (i.e., Pogostemon plectranthoides Desf., P. septentrionalis C. Y. Wu et Y. C. Huang, and two cultivars of P. cablin (Blanoco) Benth.). Comparison of all samples (including online available cp genomes of P. yatabeanus (Makino) Press and P. stellatus (Lour.) Kuntze) suggested that Pogostemon cp genomes are highly conserved in terms of genome size and gene content, with a typical quadripartite circle structure. Interspecific divergence of cp genomes has been maintained at a relatively low level, though seven divergence hotspot regions were identified by stepwise window analysis. The nucleotide diversity (Pi) value was correlated significantly with gap proportion (indels), but significantly negative with GC content. Our phylogenetic analyses based on 80 protein-coding genes yielded high-resolution backbone topologies for the Lamiaceae and Pogostemon. For the overall mean substitution rates, the synonymous (dS) and nonsynonymous (dN) substitution rate values of protein-coding genes varied approximately threefold, while the dN values among different functional gene groups showed a wider variation range. Overall, the cp genomes of Pogostemon will be useful for phylogenetic reconstruction, species delimitation and identification in the future.