The non-destructive stress measurement method is the main trend in residual stress analysis. The ultrasonic method, which utilizes the acoustoelastic effect of the longitudinal critically refracted (LCR) wave, is one of the time-saving measurement techniques. During the practical stress measurement on a workpiece, various external factors may impact the transmission of acoustic waves and the resulting stress value. This study revealed and discussed the effects of four factors on the LCR wave: surface roughness of the examined material, temperature of the material, external mechanical vibration, and surface paint. The stress coefficient was determined by comparing the offset time of the acoustic wave with the stress measured by X-ray analyzer in the zero-stress specimens, which had undergone annealing and deep cryogenic treatment. The test results indicated that the surface roughness did not affect the transition time of the acoustic wave, but it did decrease the intensity of the signal. The increase in temperature and the transition time of the acoustic wave were in a linear relationship. Mechanical vibrations from the environment would not affect the transition time or signal intensity of the acoustic wave, whereas the application of surface paint increased the transition time. Although the effect of paint on the actual workpiece could not be easily modified during stress measurement, the ultrasonic method was still suitable for monitoring the stress of a specific position of the workpiece throughout its operational lifetime. The experiment data in this study were applied to measuring the residual stress of an aluminum ship component, and the result showed a good correspondence with X-ray stress analyzer results.