These days by a high increase in the amount of computation and big data gathering and analysis, everybody needs more resources. Buying more computational and storage resources are so expensive. However, cloud computing solved this problem by providing a "pay as you go" plans therefor, users will only pay for resources that they used. However, using this technology has its challenges. One of them is resource management, which is focusing on the methodologies of dedicating resources to the users with the minimum of waste. In this paper, we propose a novel energy-aware resource management technique, using the concepts of both joint VM and container consolidation approach and deep Q-Learning algorithm for green computing in cloud data centers in order to minimize the waste of resources, migration rate, and energy.