Experiments have established that in interphase chromosomes, active and inactive loci are spatially segregated on a few mega base pairs (Mbp) scale, resulting in compartment formation. On the sub-Mbp scales, Topologically Associating Domains (TADs) appear as squares along the diagonal in the contact matrix (CM). Hi-C experiments showed that most of the TADs vanish upon deleting cohesin, while the compartment structure is maintained or even modestly enhanced. However, closer inspection of the data reveals that a non-negligible fraction of TADs is preserved (P-TADs), even after cohesin loss. Super resolution tracing, on the other hand, shows that, at the single cell level, TAD-like structures are visible after depleting cohesin. To rationalize these disparate findings, we used polymer simulations, which reveal that TADs with epigenetic mismatches across their boundaries survive after deleting the loops. Analyses of 3D structures reveal that many of the P-TADs have sharp physical boundaries. Informed by the simulations, we analyzed the Hi-C maps (with and without cohesin) in mouse liver and HCT-116, which affirmed that epigenetic mismatches and physical boundaries (calculated using the 3D structures from Hi-C contact maps) explain the origin of P-TADs. The calculated single-cell structures, using cohesin depleted Hi-C as input, have TAD-like features in single cells that are remarkably similar to the findings in imaging experiments. Cumulatively our study, utilizing a variety of methods, shows that preservation of a subset of TADs upon deleting cohesin is a robust phenomenon.