In order to improve the joint performances by controlling the thermal effect of tool shoulder, a non-shoulderplunge (NSP) welding tool is utilized to conduct friction stir welding (FSW) of an Al-Mg aluminum alloy in this paper. Sound FSW joint is successfully produced by using the NSP welding tool. In contrast to conventional FSW (C-FSW), the thermal effect of tool shoulder is effectively reduced by NSP-FSW, leading to a significant size reduction of shoulderaffected zone in the joint. Besides, the NSP-FSW joint shows finer grain structures in the stir zone and thermo-mechanically affected zone, and exhibits lower extents of grain coarsening, dislocation polygonization, and particle dissolution in the heat-affected zone. Hardness maps show that the softening region width of NSP-FSW joint is only nearly half that of C-FSW joint and the NSP-FSW joint exhibits higher hardness in the heat-affected zone than C-FSW joint. The tensile test results indicate that the joint efficiency is improved from 89 % of C-FSW joint to 97 % by the NSP-FSW process.