Chromosome rearrangements are thought to promote reproductive isolation between incipient species. However, it is unclear how often, and under what conditions, fission and fusion rearrangements act as barriers to gene flow. Here we investigate speciation between two largely sympatric fritillary butterflies,Brenthis daphneandB. ino. We use a composite likelihood approach to infer the demographic history of these species from whole genome sequence data. We then compare chromosome-level genome assemblies of individuals from each species and identify a total of nine chromosome fissions and fusions. Finally, we fit a demographic model where effective population sizes and effective migration rate vary across the genome, allowing us to quantify the effects of chromosome rearrangements on reproductive isolation. We show that chromosomes involved in rearrangements experienced less effective migration since the onset of species divergence and that genomic regions near rearrangement points have a further reduction in effective migration rate. Our results suggest that the evolution of multiple rearrangements in theB. daphneandB. inopopulations, including alternative fusions of the same chromosomes, have resulted in a reduction in gene flow. While fission and fusion of chromosomes are unlikely to be the only processes that have led to speciation between these butterflies, this study shows that these rearrangements can directly promote reproductive isolation and may be involved in speciation when karyotypes evolve quickly.