This study investigated the surface residual stress for AISI 1045 steel quenched by a YAG laser. A coaxial laser spindle was installed on a CNC machine for the experiment. The laser motion was arranged to follow the path of an extended cycloid which widened the quenching area on the steel surface. Both the temperature distribution and the residual stresses were measured by thermocouples and a portable X-ray diffractometer, respectively. When the temperature distribution was cooled down near the value of the room temperature, the residual stresses were then measured after the laser quenching process. The diffractometer used a single exposure of X-ray with a two-dimensional detector to calculate the Debye–Scherrer ring (D-S ring) for the determination of the normal and shear stresses. Different laser powers were exploited for the measurement of residual stresses, including 500, 600, 700, and 900 watts. In addition to the experiment, an analytic model for the investigation of residual stresses was built by the finite element analysis for which MSC Marc was used. The assumption for the FEA was that the laser spot had a circular shape of uniform energy distribution and the thermal–elastic–plastic model was applied to the simulation for the laser quenching process. The analytic and experimental results for the surface residual stresses had excellent consistency with a maximum difference of 10.5% from the normal stresses. The numerical results for the residual stresses also revealed that the normal stresses were compressive for the laser-quenching treatment and the shear stress could be neglected compared to the normal stress.