In this study, it is proposed that coarsening austenitic grains is a key criterion for achieving giant recovery strains in polycrystalline Fe-Mn-Si based shape memory alloys. In order to verify the hypothesis, the relationship between recovery strains and austenitic grain-sizes in cast and processed Fe-Mn-Si based shape memory alloys was investigated. The recovery strain of cast Fe-19Mn-5.5Si-9Cr-4.5Ni alloy with the coarse austenitic grains of 652 μm reached 7.7% while the recovery strain of one with the relatively small austenitic grains of 382 μm was only 5.4%. Moreover, a recovery strain of 5.9%, which is the highest previously published value for solution-treated processed Fe-Mn-Si based shape memory alloys, was obtained by coarsening the austenitic grains through only solution treatment at 1483 K for 360 min in a processed Fe-17Mn-5.5Si-9Cr-5.5Ni-0.12C alloy. However, its recovery strain was still 5.9% after thermo-mechanical treatment consisting of 10% tensile strain at room temperature and annealing at 1073 K for 30 min. This happens because annealing twins play a negative role, refining the austenitic grains, limiting the recovery strains to below 6%. In summary, coarse austenitic grains enable the achievement large recovery strains by two mechanisms. Firstly, the grains are bigger, and consequently there are fewer grain boundaries, and thus their suppressive effects of grain boundaries on stress-induced ε martensitic transformation is reduced. Secondly, coarse austenitic grains are advantageous to introduce martensite with single orientation and reduce the collisions of different martensite colonies, especially when the deformation strain is large. As such, the ceiling of recovery strains is dependent on the austenitic grain-sizes.