Spin waves and their quanta, magnons, are promising candidates for next-generation electronic devices, due to their low-power consumption and compatibility with radio-frequency-based electronic devices. For achieving magnon-based hybrid quantum systems for quantum memory and computation, the investigation of spin-wave propagation at cryogenic temperatures is highly required. In this article, we report the excitation and detection of exchange spin waves with wavelengths of tens of nanometers in an yttrium iron garnet thin film at cryogenic temperatures. We find that the exchange spin waves are unidirectional in all temperature ranges, owing to the chiral dynamical dipolar coupling between the spin-wave mode in the yttrium iron garnet and the ferromagnetic resonance mode in the cobalt nanowire. Notably, a high exchange spin-wave group velocity of 2 km/s at 10 K is observed. Our results are promising for the development of high-speed and energy-efficient quantum magnonic devices operating at cryogenic temperatures.