Nanophotonic engineering provides an effective platform to manipulate thermal emission on-demand, enabling unprecedented heat management superior to conventional bulk materials. Amongst a plethora of nanophotonic structures, symmetries play an important role in controlling radiative heat transfer in both near-field and far-field. In physics, broken symmetries generally increase the degree of freedom in a system, enriching the understanding of physical mechanisms and bringing many exciting opportunities for novel applications. In this review, we discussed the underlying physics and functionalities of nanophotonic structures with broken geometrical symmetries, engineered mode symmetries, and broken reciprocity for the control of thermal emission. We overview a variety of physical phenomena and interesting applications, and provide the outlook for future development.