Gas turbine vibration data may exhibit considerable differences under time-varying conditions, which poses challenges for neural network anomaly detection. We first propose a framework for a gas turbine vibration frequency spectra process under time-varying operation conditions, assisting neural networks’ ability to capture weak information. The framework involves scaling spectra for aligning all frequency components related to rotational speed and normalizing frequency amplitude in a self-adaptive way. Degressive beta variational autoencoder is employed for learning spectra characteristics and anomaly detection, while a multi-category anomaly index is proposed to accommodate various operating conditions. Finally, a dataset of blade Foreign Object Damage (FOD) fault occurring under time-varying operating conditions was used to validate the framework and anomaly detection. The results demonstrate that the proposed method can effectively reduce the spectra differences under time-varying conditions, and also detect FOD fault during operation, which are challenging to identify using conventional methods.