The axion is a well-motivated candidate for the inflaton, as the radiative corrections that spoil many single-field models are avoided by virtue of its shift symmetry. However, axions generically couple to gauge sectors. As the axion slow-rolls during inflation, this coupling can cause the production of a non-diluting thermal bath, a situation known as "warm inflation." This thermal bath can dramatically alter inflationary dynamics and observable predictions. In this paper, we demonstrate that a thermal bath can form for a wide variety of initial conditions, including starting from zero initial temperature in the universe. Furthermore, we find that axion inflation becomes warm over a large range of couplings, and explicitly map the parameter space for two axion inflation potentials. We show that in large regions of parameter space, axion inflation models once assumed to be safely "cold" are in fact warm, and must be reevaluated in this context.