Topologically associating domains (TADs) are regarded as functional and structural units of higher-order spatial genome organization of many eukaryotic genomes. However, our knowledge of how evolution affects TADs remains limited. To decipher the evolutionary significance of TADs, we de novo assembled the genome of D. pseudoobscura and created a high-resolution (~800 bp) Hi-C contact map to annotate TADs. Remarkably, more than 40% of TADs between D. pseudoobscura and D. melanogaster are conserved, despite extensive chromosomal rearrangement in the ~49 million years since they shared a common ancestor. Comparison of 17 diverse Drosophila species genomes revealed enrichment of genome rearrangement breakpoints at the TAD boundaries but depletion of such breaks inside the TADs themselves. We show that conservation of TADs is associated with gene expression stability across tissues. Surprisingly, despite being larger mutational targets, a substantial proportion of long (>50kb) genes in D. melanogaster (42%) and D. pseudoobscura (26%) are individually spanned by complete TADs, implying the formation and maintenance of TADs via 3D cis-regulatory interactions commonly found within long genes. U sing high-confidence genome-wide structural variant datasets from 14 D. melanogaster strains, its 3 closest sibling species from the D. simulans species complex, and two obscura clade species, we show evidence of natural selection operating on structural variants at the TAD boundaries, but with the nature of selection differing between the SV types. Deletions are significantly depleted at TAD boundaries for both divergent and polymorphic SVs, suggesting that deletions at TAD boundaries are under purifying selection, whereas divergent duplications are enriched at the TAD boundaries, pointing to positive selection. Our results offer novel insights into the evolutionary role and maintenance of TADs and their significance in genome structure evolution and gene regulation.