Integrated space-air-ground networks promise to offer a valuable solution space for empowering the sixth generation of communication networks (6G), particularly in the context of connecting the unconnected and ultraconnecting the connected. Such digital inclusion thrive makes resource management problems, especially those accounting for load-balancing considerations, of particular interest. The conventional modelbased optimization methods, however, often fail to meet the realtime processing and quality-of-service needs, due to the high heterogeneity of the space-air-ground networks, and the typical complexity of the classical algorithms. Given the premises of artificial intelligence at automating wireless networks design, this paper focuses on showcasing the prospects of machine learning in the context of user scheduling in integrated space-air-ground communications. The paper first overviews the most relevant state-of-the art in the context of machine learning applications to the resource allocation problems, with a dedicated attention to space-air-ground networks. The paper then proposes, and shows the benefit of, one specific application that uses ensembling deep neural networks for optimizing the user scheduling policies in integrated space-high altitude platform station (HAPS)-ground networks. Finally, the paper sheds light on the challenges and open issues that promise to spur the integration of machine learning in space-air-ground networks, namely, online HAPS power adaptation, learning-based channel sensing, data-driven multi-HAPSs resource management, and intelligent flying taxisempowered systems.