This paper presents the results of measuring the Rayleigh wave as well as longitudinal wave velocities in a concrete specimen using transient elastic waves. The Rayleigh surface wave generated by a steel ball impact on a concrete material is studied in detail and then a method for the determination of Rayleigh surface wave velocity based on the cross-correlation method is proposed. The longitudinal wave velocity of a concrete specimen is determined by measuring the wavefront arrival of a longitudinal wave. A special triggering device for the accurate determination of the impact time origin is utilized. The measured velocities of both the Rayleigh wave and the longitudinal wave in the concrete specimen are in good agreement with those measured by utilizing the conventional ultrasonic method. It is noted that with the Rayleigh wave and the longitudinal wave velocities measured, the Young’s modulus as well as the shear modulus of a concrete specimen can be obtained in a straightforward way.