In this study, two different types of heterogeneous structures were prepared by controlling different surface mechanical attrition treatment (SMAT) time (5 s, 10 s, 30 s, 60 s). The effect of heterogeneous Cu with bimodal grain size structure (BGSS) and gradient structure (GS) on heterogeneous deformation induced (HDI) stress and mechanical properties was investigated systematically. By combining surface morphology with microhardness distribution, it was found that the surface grains of the SMAT-ed Cu (5 s and 10 s) were partially refined to exhibit BGSS, while the surface grains of SMAT-ed Cu (30 s and 60 s) were sufficiently refined to form GS. The load-unload-reload (LUR) tests results showed that HDI stress for SMAT 60s Cu was much higher than that of the SMAT 10s Cu. Furthermore, microstructure characterizations revealed that SMAT-ed Cu with BGSS and GS suppressed strain localization, which resulted in high strength and reasonable ductility.