To achieve high quality welding of an AZ31 magnesium alloy sheet, butt experiments are conducted using an Nd:YAG pulsed laser welder to weld the AZ31 magnesium alloy (1 mm). The influence of preheating temperature on the microscopic structure and mechanical performance of AZ31 magnesium alloy pulsed laser welding is studied. Experimental results show that with the rising of preheating temperature, the grain size of an equiaxed crystal in the weld center and columnar crystal at the fusion boundary (FB) first decreases and then increases and reaches the minimum values of 18.89 ± 0.45 and 32.09 ± 1.52 μm, respectively, when preheated at 200 °C. The maximum force and tensile strength increase first and decrease and reach the maximum of 2.32 kN and 193.3 MPa, respectively, when the preheating temperature is 300 °C. Above all, the welds with excellent mechanical performance can be obtained at the preheating temperature of 300 °C.