The prediction of stocks is complicated by the dynamic, complex, and chaotic environment of the stock market. Investors put their money into the financial market, hoping to maximize profits by understanding market trends and designing trading strategies at the entry and exit points. Most studies propose machine learning models to predict stock prices. However, constructing trading strategies is helpful for traders to avoid making mistakes and losing money. We propose an automatic trading framework using LSTM combined with deep Q-learning to determine the trading signal and the size of the trading position. This is more sophisticated than traditional price prediction models. This study used price data from the Taiwan stock market, including daily opening price, closing price, highest price, lowest price, and trading volume. The profitability of the system was evaluated using a combination of different states of different stocks. The profitability of the proposed system was positive after a long period of testing, which means that the system performed well in predicting the rise and fall of stocks.