The prediction of the stock price index is a challenge even with advanced deep-learning technology. As a result, the analysis of volatility, which has been widely studied in traditional finance, has attracted attention among researchers. This paper presents a new forecasting model that combines asymmetric fractality and deep-learning algorithms to predict a one-day-ahead absolute return series, the proxy index of stock price volatility. Asymmetric Hurst exponents are measured to capture the asymmetric long-range dependence behavior of the S&P500 index, and recurrent neural network groups are applied. The results show that the asymmetric Hurst exponents have predictive power for one-day-ahead absolute return and are more effective in volatile market conditions. In addition, we propose a new two-stage forecasting model that predicts volatility according to the magnitude of volatility. This new model shows the best forecasting performance regardless of volatility.