We study the effects of diffusion on a -gradient echo memory, which is a coherent optical quantum memory, using thermal gases. The efficiency of this memory is high for short storage time, but decreases exponentially due to decoherence as the storage time is increased. We study the effects of both longitudinal and transverse diffusion in this memory system, and give both analytical and numerical results that are in good agreement. Our results show that diffusion has a significant effect on the efficiency. Further, we suggest ways to reduce these effects to improve storage efficiency. We also report on a mechanism by which the rate of expansion of the transverse width of the beam is reduced compared to the naive expectation of diffusive effects, as observed in recent experiments.