We consider the ground state and the ground state energy of an atom with spinless electrons in the framework of non-relativistic quantum electrodynamics. We show that the ground state energy as well as the ground state depend analytically on the parameters of the group of dilations, the parameter of a group of spatial dependent phase changes, and on the minimal coupling constant. As a corollary we obtain spatial exponential decay of the ground state as well as of its dilation analytic extension. No infrared regularization is needed for the result. Our result is based on operator theoretic renormalization.