The concept of multi-principal component has created promising opportunities for the development of novel high-entropy ceramics (HEC) for extreme environments encountered in advanced turbine engines, nuclear reactors, and hypersonic vehicles, as it expands the compositional space of ceramic materials with tailored properties within a single-phase solution. Some unique physical properties of HECs, such as higher hardness, high-temperature strength, lower thermal conductivity, and improved irradiation resistance than the constitute ceramics, have been observed. The promising properties of HECs are attributed to the compositional complexity, atomic-level disorder, lattice distortion, and other fundamental processes related to defect formation and phonon scattering. This manuscript serves as a critical review of the recent progress in high-entropy carbides and borides, focusing on synthesis and evaluations of their performance in extreme high-temperature, irradiation, and gaseous environments.