To improve the signal-to-noise ratio (SNR) of vibration signals in a phase-sensitive optical time-domain reflectometer (Φ-OTDR) system, a principal component analysis variable step-size normalized least mean square (PCA-VSS-NLMS) denoising method was proposed in this study. First, the mathematical principle of the PCA-VSS-NLMS algorithm was constructed. This algorithm can adjust the input signal to achieve the best filter effect. Second, the effectiveness of the algorithm was verified via simulation, and the simulation results show that compared with the wavelet denoising (WD), Wiener filtering, variational mode decomposition (VMD), and variable step-size normalized least mean square (VSS-NLMS) algorithms, the PCA-VSS-NLMS algorithm can improve the SNR to 30.68 dB when the initial SNR is −1.23 dB. Finally, the PCA-VSS-NLMS algorithm was embedded into the built Φ-OTDR system, an 11.22 km fiber was measured, and PZT was added at 10.19–10.24 km to impose multiple sets of fixed-frequency disturbances. The experimental results show that the SNR of the vibration signal is 8.77 dB at 100 Hz and 0.07 s, and the SNR is improved to 26.17 dB after PCA-VSS-NLMS filtering; thus, the SNR is improved by 17.40 dB. This method can improve the SNR of the system’s position information without the need to change the existing hardware conditions, and it provides a new scheme for the detection and recognition of long-distance vibration signals.