The advent of the wireless communications systems augurs new cutting-edge technologies, including self-driving vehicles, unmanned aerial systems, autonomous robots, the Internet of-Things, and virtual reality. These technologies require high data rates, ultra-low latency, and high reliability, all of which are promised by the fifth generation of wireless communication systems (5G). Many research groups state that 5G cannot meet its demands without artificial intelligence (AI) integration as 5G wireless networks are expected to generate unprecedented traffic giving wireless research designers access to big data that can help in predicting the demands and adjust cell designs to meet the users' requirements. Subsequently, many researchers applied AI in many aspects of 5G wireless communication design including radio resource allocation, network management, and cyber-security. In this paper, we provide an in-depth review of AI for 5G wireless communication systems. In this respect, the aim of this paper is to survey AI in 5G wireless communication systems by discussing many case studies and the associated challenges, and shedding new light on future research directions for leveraging AI in 5G wireless communications.