α-Zirconium phosphate (Zr(HPO 4 ) 2 ⋅H 2 O, α-ZrP) is an inorganic layered compound. Since the first report of crystalline α-ZrP in 1964, thanks to its simple synthesis and unique physiochemical properties, α-ZrP has found widespread application in many fields including mechanical reinforcing, barrier improvement, flame retardancy, anticorrosion, catalysis, environment, energy, and medicine. Because of the ease of exfoliation of α-ZrP to obtain single-layer nanosheets, as well as its rich surface chemistry thanks to the high density of orderly arranged surface acidic hydroxyl groups, α-ZrP single-layer nanosheets are ideal building blocks for self-assembly or assembly with other chemicals. The assembly of α-ZrP nanosheets could form a randomly dispersed structure (isotropic), roughly ordered structure (nematic), or highly ordered structure (smectic) within liquid colloids or solid hybrids (including hydrogels). Thanks to the combination of the unique structures and the novel functions of the components, the assembled materials have found a wide verity of applications due to their excellent properties. In this article, the methods to synthesize α-ZrP, the approaches and mechanisms to exfoliate α-ZrP, and the strategies to assemble α-ZrP nanosheets to form various structures, as well as the applications of the assembled materials are reviewed. The emerging prospects of α-ZrP nanosheets as a key material in next-generation functional applications are envisioned.