We study the performance of gaseous and two-phase (liquid-gas) cryogenic detectors of ionizing radiation, based on gas electron multipliers (GEMs). For the first time, relatively high gas gains, exceeding 10 , were obtained in pure He, Ar, and Kr at low temperatures and the stable avalanche mode of operation was observed in a two-phase cryogenic detector in Kr. The electron avalanching at low temperatures, in the range of 120-300 K, is systematically studied.