Plasmids incur a fitness cost that has the potential to restrict the dissemination of resistance in bacterial pathogens. However, bacteria can overcome this disadvantage by compensatory evolution to maintain their resistance. Compensatory evolution can occur via both chromosomes and plasmids, but there are a few reviews regarding this topic, and most of them focus on plasmids. In this review, we provide a comprehensive overview of the currently reported mechanisms underlying compensatory evolution on chromosomes and plasmids, elucidate key targets regulating plasmid fitness cost, and discuss future challenges in this field. We found that compensatory evolution on chromosomes primarily arises from mutations in transcriptional regulatory factors, whereas compensatory evolution of plasmids predominantly involves three pathways: plasmid copy number regulation, conjugation transfer efficiency, and expression of antimicrobial resistance (AMR) genes. Furthermore, the importance of reasonable selection of research subjects and effective integration of diverse advanced research methods is also emphasized in our future study on compensatory mechanisms. Overall, this review establishes a theoretical framework that aims to provide innovative ideas for minimizing the emergence and spread of AMR genes.