Communication networks are expanding rapidly and becoming increasingly complex. As a consequence, the conventional rule-based algorithms or protocols may no longer perform at their best efficiencies in these networks. Machine learning (ML) has recently been applied to solve complex problems in many fields, including finance, health care, and business. ML algorithms can offer computational models that can solve complex communication network problems and consequently improve performance. This paper reviews the recent trends in the application of ML models in communication networks for prediction, intrusion detection, route and path assignment, Quality of Service improvement, and resource management. A review of the recent literature reveals extensive opportunities for researchers to exploit the advantages of ML in solving complex performance issues in a network, especially with the advancement of softwaredefined networks and 5G.