Spin-transfer magnetic random access memory is of significant interest for cryogenic applications where a persistent, fast, low-energy consumption and high device density is needed. Here we report the low-temperature nanosecond duration spin-transfer switching characteristics of perpendicular magnetic tunnel junction (pMTJ) nanopillar devices (40 to 60 nm in diameter) and contrast them to their room temperature properties. Interestingly, at fixed pulse voltage overdrive the characteristic switching time decreases with temperature, in contrast to macrospin model predictions, with the largest reduction in switching time occurring between room temperature and 150 K. The switching energy increases with decreasing temperature, but still compares very favorably to other types of spin-transfer devices at 4 K, with < 300 fJ required per switch. Write error rate (WER) measurements show highly reliable (WER ≤ 5×10 -5 with 4 ns pulses at 4 K) demonstrating the promise of pMTJ devices for cryogenic applications and routes to further device optimization.