Ferroelectric (FE) materials, which typically adopt the perovskite structure with non-centrosymmetry and exhibit spontaneous polarization, are promising for applications in memory, electromechanical and energy storage devices. However, these advanced applications suffer from the intrinsic limitations of perovskite FEs, including poor complementary metal oxide semiconductor (CMOS) compatibility and environmental issues associated with lead. Hafnium oxide (HfO2), with stable bulk centrosymmetric phases, possesses robust ferroelectricity in nanoscale thin films due to the formation of non-centrosymmetric phases. Owing to its high CMOS compatibility and high scalability, HfO2 has attracted significant attention. In the last decade, significant efforts have been made to explore the origin of the ferroelectricity and factors that influence the FE properties in HfO2 films, particularly regarding the role of microstructure, which is vital in clarifying these issues. Although several comprehensive reviews of HfO2 films have been published, there is currently no review focused on the relationship between microstructure and FE properties. This review focuses on the microstructure-property relationships in FE polycrystalline and epitaxial HfO2 films. The crystallographic structures and characterization methods for HfO2 polymorphs are first discussed. For polycrystalline HfO2 films, the microstructure-FE properties relationships, driving force and kinetic pathway of phase transformations under growth parameters or external stimuli are reviewed. For epitaxial films, the lattice matching relations between HfO2 films and substrates and the corresponding impact on the FE properties are discussed. The FE properties between polycrystalline and epitaxial HfO2 films are compared based on their different microstructural characteristics. Finally, a future perspective is given for further investigating FE HfO2 films.