Workforce management is one of several critical issues in a call center. A call center supervisor must assign an adequate number of call agents to handle a high volume of time-variant incoming calls. Without effective staff allocation, improper workforce management can degrade service quality and reduce customer satisfaction. This paper presents a novel call center workforce management based on a deep neural network and reinforcement learning (RL). The proposed method first uses a deep neural network to learn and predict call center traffic characteristics. The deep neural network consists of a Long-Short Term Memory (LSTM) network and a Deep Neural Network (DNN) capturing non-linear call traffic behaviors. The expected traffic parameters are supplied into the Erlang A model, which calculates important service metrics, including a call abandonment probability and the average response time. This paper applies a reinforcement learning framework using the Q-learning algorithm to establish the optimal starting times of call agent shifts and their associated call agent numbers by maximizing a defined reward function to handle dynamic call center traffic. The objective of these findings is to maintain the quality of service of a call center throughout working hours. The proposed method surpasses experienced human supervisors and previous workforce management schemes in terms of achieved qualities of service and average waiting time from experimental results under actual call center data.