Wireless communication networks have been witnessing unprecedented demand due to the increasing number of connected devices and emerging bandwidth-hungry applications. Although there are many competent technologies for capacity enhancement purposes, such as millimeter wave communications and network densification, there is still room and need for further capacity enhancement in wireless communication networks, especially for the cases of unusual people gatherings, such as sport competitions, musical concerts, etc. Unmanned aerial vehicles (UAVs) have been identified as one of the promising options to enhance capacity due to their easy implementation, pop-up fashion operation, and cost-effective nature. The main idea is to deploy base stations on UAVs and operate them as flying base stations, thereby bringing additional capacity where it is needed. However, UAVs mostly have limited energy storage, hence, their energy consumption must be optimized to increase flight time. In this survey, we investigate different energy optimization techniques with a top-level classification in terms of the optimization algorithm employed—conventional and machine learning (ML). Such classification helps understand the state-of-the-art and the current trend in terms of methodology. In this regard, various optimization techniques are identified from the related literature, and they are presented under the above-mentioned classes of employed optimization methods. In addition, for the purpose of completeness, we include a brief tutorial on the optimization methods and power supply and charging mechanisms of UAVs. Moreover, novel concepts, such as reflective intelligent surfaces and landing spot optimization, are also covered to capture the latest trends in the literature.