Ternary (I-III-VI) and quaternary (I-II-IV-VI) metal-chalcogenides like CuInS2 or Cu2ZnSn(S,Se)4 are among the materials currently most intensively investigated for various applications in the area of alternative energy conversion and light-emitting devices. They promise more sustainable and affordable solutions to numerous applications, compared to more developed and well understood II-VI and III-V semiconductors. Potentially superior properties are based on an unprecedented tolerance of these compounds to non-stoichiometric compositions and polymorphism. However, if not properly controlled, these merits lead to undesirable coexistence of different compounds in a single polycrystalline lattice and huge concentrations of point defects, becoming an immense hurdle on the way toward real-life applications. Raman spectroscopy of phonons has become one of the most powerful tools of structural diagnostics and probing physical properties of bulk and microcrystalline I-III-VI and I-II-IV-VI compounds. The recent explosive growth of the number of reports on fabrication and characterisation of nanostructures of these compounds must be pointed out as well as the steady use of Raman spectroscopy for their characterisation. Interpretation of the vibrational spectra of these compound nanocrystals (NCs) and conclusions about their structure can be complicated compared to bulk counterparts because of size and surface effects as well as emergence of new structural polymorphs that are not realizable in the bulk. This review attempts to summarise the present knowledge in the field of I-III-VI and I-II-IV-VI NCs regarding their phonon spectra and capabilities of Raman and IR spectroscopies in the structural characterisations of these promising families of compounds.