This paper provides a comprehensive overview of the evolution of Machine Learning (ML), from traditional to advanced, in its application and integration into unmanned aerial vehicle (UAV) communication frameworks and practical applications. The manuscript starts with an overview of the existing research on UAV communication and introduces the most traditional ML techniques. It then discusses UAVs as versatile actors in mobile networks, assuming different roles from airborne user equipment (UE) to base stations (BS). UAV have demonstrated considerable potential in addressing the evolving challenges of next-generation mobile networks, such as enhancing coverage and facilitating temporary hotspots but pose new hurdles including optimal positioning, trajectory optimization, and energy efficiency. We therefore conduct a comprehensive review of advanced ML strategies, ranging from federated learning, transfer and meta-learning to explainable AI, to address those challenges. Finally, the use of stateof-the-art ML algorithms in these capabilities is explored and their potential extension to cloud and/or edge computing based network architectures is highlighted.