The path-dependent magnetizations (polarizations) and susceptibilities in dipole-and spin-glasses are described analytically for the standard protocols of temperature and field variations using the phenomenological Landau-type description of multiple metastable states in the nonergodic phases. The immediate manifestation of metastable states' multiplicity as staircases of magnetizations (polarizations) curves in the temperature cycling experiments is explained and described. The obtained results are in a reasonable qualitative agreement with the existing experimental data.