Smart grids represent a paradigm shift in the electricity industry, moving from traditional one‐way systems to more dynamic, interconnected networks. These grids are characterised by their intelligent automation, robust structure, and enhanced interaction with customers, backed by comprehensive monitoring and data analytics. The key of this transformation is the integration of data‐driven methods into smart grids. Compared to previous big data solutions, large language models (LLMs), with their advanced generalisation abilities and multi‐modal competencies, are crucial in effectively managing and integrating diverse data sources. They address challenges such as data inconsistency, inadequate quality, and heterogeneity, thereby enhancing the operational efficiency and reliability of smart grids. Furthermore, at the system level, LLMs improve human–system interactions, making smart grids more user‐friendly and intuitive. Last but not the least, the structure of LLMs performs inherent advantages in bolstering system security and privacy, alongside in resolving issues related to system compatibility and integration. The paper reviews the data‐empowered smart grids and for the first time finds and proposes opportunities and future directions for adopting LLMs to accelerate the mass‐scale application of Smart Grids.