Modern wrought magnesium alloys have poor formability at room temperature, which impedes the wider uptake of these alloys by the industry. Over the last decades, research activities in the area of magnesium alloy development have grown enormously and have produced a pallet of exciting findings, which the authors summarize in this concise review focused on the effect of the microstructure, primarily of grain size, on room temperature ductility of wrought Mg-based alloys. Well-established paths and modern strategies to control over the grain size distribution are discussed. It is demonstrated that the use of severe plastic deformation techniques for ultimate grain refinement in magnesium alloys opens new windows for improving their mechanical properties profile by managing both strength and ductility in a wide range. However, it is shown that grain size alone cannot be regarded as a key parameter controlling the mechanical behavior of Mg alloys. The effect of texture is of paramount importance for the overall mechanical response of Mg alloys and this may supersede the influence of grain size.