Cs3(HSO4)2(H2PO4) and Cs4(HSO4)3(H2PO4) single crystals were obtained for the first time based on a systematic study of the phase diagram of the CsHSO4-CsH2PO4-H2O ternary system. To reveal the structural conditionality for anomalies in physical properties, including dielectric and optical properties, diffraction studies of the crystal structure of Cs3(HSO4)2(H2PO4) and Cs4(HSO4)3(H2PO4) were performed, and their common structural features and differences were analyzed.