In the present work, it is shown that the surface preparation method used on two Ni-based superalloys prior to aluminizing chemical vapor deposition (CVD) is one of the most important factors determining the oxidation resistance of aluminized Ni-based superalloys. It was found that grit-blasting the substrate surface negatively affects the oxidation resistance of the aluminized coatings. For grit-blasted and aluminized IN 625, a thicker outer NiAl coating was formed compared to that of IN 738. In contrast, no effect on NiAl coating thickness was found for grit-blasted and aluminized IN 738. However, a thicker interdiffusion zone (IDZ) was observed. It was shown that the systems with grit-blasted surfaces reveal worse oxidation resistance during thermal shock tests—namely, a higher mass loss was observed for both grit-blasted and aluminized alloys, as compared to ground and aluminized alloys. A possible reason for this effect of remaining alumina particles originating from surface grit-blasting on the diffusion processes and stress distribution at the coating/substrate is proposed.