Compositionally complex materials have demonstrated extraordinary promise for structural robustness in extreme environments. Of these, the most commonly thought of are high entropy alloys, where chemical complexity grants uncommon combinations of hardness, ductility, and thermal resilience. In contrast to these metal-metal bonded systems, the addition of ionic and covalent bonding has led to the discovery of high entropy ceramics. These materials also possess outstanding structural, thermal, and chemical robustness but with a far greater variety of functional properties which enable access to continuously controllable magnetic, electronic, and optical phenomena. In this experimentally focused perspective, we outline the potential for high entropy ceramics in functional applications under extreme environments, where intrinsic stability may provide a new path toward inherently hardened device design. Current works on high entropy carbides, actinide bearing ceramics, and high entropy oxides are reviewed in the areas of radiation, high temperature, and corrosion tolerance where the role of local disorder is shown to create pathways toward self-healing and structural robustness. In this context, new strategies for creating future electronic, magnetic, and optical devices to be operated in harsh environments are outlined.