The heterogeneous market hypothesis states that there are different types of investors with different investment term-structures in the stock market. Based on the hypothesis, we analyze the trading strategies of moving averages with different term structures, and we find that the trading strategies formed by the 1-month short-term moving average with 3-, 6-, 9-, 12-, 18-, and 24-month long-term moving averages can gain significant excess returns in the Chinese market. Accordingly, this paper adopts the moving average factor formed by the method of Liu’s to expand the current four-factor pricing model of the Chinese market. The results of spanning regression show that the introduction of the short-term average with a period of 1-month and the long-term moving average with a period of 3- and 12-month can significantly improve the pricing power. The GRS test developed by Gibbons et al. (1989) also shows that the augmented six-factor model with moving average factors can pass the test at the significance level of 5%, and the average absolute value of intercept terms decreases by about 50% compared with the results of the four-factor model. At the same time, under different macro-state dependence, the overall performance of the augmented six-factor model with moving averages is still better than that of the four-factor pricing model proposed by Liu et al.’s. Our main contribution is to introduce double moving average factors to capture the behaviors of investors’ different term structures and add these factors to the most competitive asset pricing model for enhancing the pricing power. It is of great significance to supplement the pricing model in the Chinese market.