Polymer thin films doped with azobenzene molecules do have the ability to organize themselves in spontaneous surface relief gratings (SRG) under irradiation using a single polarized beam. To shed some light on this still unexplained phenomenon, we use a new method that permits us to access experimentally the very first steps of the pattern formation process. By decreasing the temperature, we slow down the formation and organization of patterns, due to the large increase in the viscosity and relaxation time of the azopolymer. As a result, decreasing the temperature allows us to access and study much shorter time scales, in the physical mechanisms underlying the pattern formation, than those previously reported. We find that the patterns organize themselves in sub-structures which size increases with the temperature, following the diffusion coefficient evolution of the material. This result suggests that the pattern formation and organization are mainly governed by diffusive processes, in agreement with some theories of SRG formation. Upon decreasing the temperature further, we observe the emergence of small voids located at the junction of the sub-structures.