High-entropy ceramics (HECs) are solid solutions of inorganic compounds with one or more Wyckoff sites shared by equal or near-equal atomic ratios of multi-principal elements. Although in the infant stage, the emerging of this new family of materials has brought new opportunities for material design and property tailoring. Distinct from metals, the diversity in crystal structure and electronic structure of ceramics provides huge space for properties tuning through band structure engineering and phonon engineering. Aside from strengthening, hardening, and low thermal conductivity that have already been found in high-entropy alloys, new properties like colossal dielectric constant, super ionic conductivity, severe anisotropic thermal expansion coefficient, strong electromagnetic wave absorption, etc., have been discovered in HECs. As a response to the rapid development in this nascent field, this article gives a comprehensive review on the structure features, theoretical methods for stability and property prediction, processing routes, novel properties, and prospective applications of HECs. The challenges on processing, characterization, and property predictions are also emphasized. Finally, future directions for new material exploration, novel processing, fundamental understanding, in-depth characterization, and database assessments are given.