Spectral denoising based on Hilbert–Huang transform combined with F-test

Abstract

Due to the influence of uncontrollable factors such as the environment and instruments, noise is unavoidable in a spectral signal, which may affect the spectral resolution and analysis result. In the present work, a novel spectral denoising method is developed based on the Hilbert–Huang transform (HHT) and F-test. In this approach, the original spectral signal is first decomposed by empirical mode decomposition (EMD). A series of intrinsic mode functions (IMFs) and a residual (r) are obtained. Then, the Hilbert transform (HT) is performed on each IMF and r to calculate their instantaneous frequencies. The mean and standard deviation of instantaneous frequencies are calculated to further illustrate the IMF frequency information. Third, the F-test is used to determine the cut-off point between noise frequency components and non-noise ones. Finally, the denoising signal is reconstructed by adding the IMF components after the cut-off point. Artificially chemical noised signal, X-ray diffraction (XRD) spectrum, and X-ray photoelectron spectrum (XPS) are used to validate the performance of the method in terms of the signal-to-noise ratio (SNR). The results show that the method provides superior denoising capabilities compared with Savitzky–Golay (SG) smoothing.

Intelligent Fault Diagnosis Method Based on VMD-Hilbert Spectrum and ShuffleNet-V2: Application to the Gears in a Mine Scraper Conveyor Gearbox

“…Therefore, before the Hilbert transform of IMF, it is necessary to judge the correlation degree between each IMF component and the diagnosed fault to ensure the accuracy of fault feature extraction and the effectiveness of diagnosis. The sensitive IMF discrimination method combines the correlation between each IMF and fault signal with the correlation of the normal signal, and the combination of Hilbert transform can highlight the fault information and weaken the influence of the normal information [ 38 ].…”

The application of generalized S-transform in the denoising of surface plasmon resonance (SPR) spectrum

“…Step 2: treat the SPR reection spectrum as a time series signal, and perform time-frequency analysis on the spectral data based on the generalized S-transform to obtain the timefrequency distribution of the signal by using eqn (7).…”

“…In addition, common denoising methods include Smoothing filtering, fast Fourier transform (FFT) filtering, wavelet transform filtering, and empirical mode decomposition (EMD) filtering. Among them, smooth filtering is generally completed by using the averaging method, 7 which averages the neighboring points to suppress noise. The size of the neighborhood is directly related to the smoothing effect.…”

Comparison of various chemometric methods on visible and near-infrared spectral analysis for wood density prediction among different tree species and geographical origins

“…Pre-processing is an essential step for improving model prediction accuracy through converting raw spectral data into a new data set without interferences (Bian et al, 2020). The common used pre-processing techniques are multiplicative scatter correction (MSC), the first derivative, Savitzky-Golay (SG) filtering, detrending (DT), wavelet transform (WT) and standard normal variate (SNV) (Dotto et al, 2018;Li et al, 2020;Bian et al, 2022;Carvalho et al, 2022;Ling et al, 2022). Different results will be obtained using various pre-processing methods or their combinations due to the different mechanism and functions, thus, it is important to select the most useful method and prevent the phenomena of over-fitting.…”