Thermal management is critical for safety, performance and durability of lithium-ion batteries that are ubiquitous in consumer electronics, electric vehicles (EVs), aerospace, and grid-scale energy storage. Towards mass adoption of EVs globally, lithium-ion batteries are increasingly used under extreme conditions including low temperatures, high temperatures and fast charging. Furthermore, EV fires caused by battery thermal runaway have become a major hurdle. These extreme conditions pose great challenges for thermal management and require unconventional strategies. The interactions between thermal, electrochemical, materials and structural characteristics of batteries further complicate the challenges, but also enable opportunities for developing innovative strategies. In this review, the challenges for thermal management under extreme conditions are analyzed. Then the progress is highlighted in two directions. One is improving battery thermal management systems based on the principles of heat transfer, which are generally external to Li-ion cells. The other is designing novel battery structures, which are generally internal of Li-ion cells such as smart batteries with embedded sensors and actuators. The latter approach could greatly simplify or even eliminate the need for battery thermal management under extreme conditions. New research integrating these two approaches is recommended.