The commonly used baseline-free Lamb wave damage identification methods often require a large amount of sensor data to eliminate the dependence on baseline signals. To improve the efficiency of damage localization, this paper proposes a new Lamb wave damage location method, namely the probabilistic exchanging-element time-reversal method (PEX-TRM), which is based on the exchanging-element time-reversal method (EX-TRM) and the probabilistic damage identification method. In this method, the influence of the damage wave packet migration on the correlation coefficient between the reconstructed signals of each sensing path and the initial excitation signal is analyzed, and the structure is divided into multiple regional units corresponding to the damage to locate damage. In addition, the influence of the number of sensing paths on the location accuracy is also analyzed. A method of damage probability imaging based on structural symmetry is proposed to enhance location accuracy in the case of sparse sensing paths. The experimental and simulation results verify that the method can achieve damage location with fewer excitation times. Moreover, this method can avoid the problem that the damage wave packet is difficult to extract, improve the efficiency of damage location, and promote the engineering application of the Lamb wave damage location method.