The autoregressive moving average (ARMA) model takes the significant position in time series analysis for a wide-sense stationary time series. Facing the trend and seasonal component of a time series, the difference operator and seasonal difference operator, which are bases of ARIMA and SARIMA, respectively, are introduced to remove the trend and seasonal component so that the original non-stationary time series could be transformed into a wide-sense stationary time series, which could then be handled by Box-Jenkins methodology (ARMA). However, such difference operators are more practical experiences than exact theories by now. In this paper, we investigate the power of the (seasonal) difference operator from the perspective of spectral analysis, linear system theory and digital filtering, and point out the characteristics and limitations of (seasonal) difference operator. Besides, the general method that transforms a non-stationary (the non-stationarity in the mean sense) stochastic process to be wide-sense stationary will be presented.