Phonocardiogram (PCG), the graphic recording of heart signals, is analyzed to determine the cardiac mechanical function. In the recording of PCG signals, the major problem encountered is the corruption by surrounding noise signals. The noise-corrupted signal cannot be analyzed and used for advanced processing. Therefore, there is a need to denoise these signals before being employed for further processing. Adaptive Noise Cancellers are best suited for signal denoising applications and can efficiently recover the corrupted PCG signal. This paper introduces an optimal adaptive filter structure using a Sign Error LMS algorithm to estimate a noise-free signal with high accuracy. In the proposed filter structure, a noisy signal is passed through a multistage cascaded adaptive filter structure. The number of stages to be cascaded and the step size for each stage are adjusted automatically. The proposed Variable Stage Cascaded Sign Error LMS (SELMS) adaptive filter model is tested for denoising the fetal PCG signal taken from the SUFHS database and corrupted by Gaussian and colored pink noise signals of different input SNR levels. The proposed filter model is also tested for pathological PCG signals in the presence of Gaussian noise. The simulation results prove that the proposed filter model performs remarkably well and provides 8–10 dB higher SNR values in a Gaussian noise environment and 2-3 dB higher SNR values in the presence of colored noise than the existing cascaded LMS filter models. The MSE values are improved by 75–80% in the case of Gaussian noise. Further, the correlation between the clean signal and its estimate after denoising is more than 0.99. The PSNR values are improved by 7 dB in a Gaussian noise environment and 1-2 dB in the presence of pink noise. The advantage of using the SELMS adaptive filter in the proposed filter model is that it offers a cost-effective hardware implementation of Adaptive Noise Canceller with high accuracy.