The collection and storage of large-scale load data in a smart grid provide new approaches for the efficient, economical, and safe operation of power systems.Deep Learning (DL) has become increasingly popular for large-scale load data analytics in recent years because of its ability to extract latent features and discovering complex relationships. This paper first overviews eight typical open load datasets of the grid and smart meter collected worldwide, the challenges faced by conventional machine learning, and the DL techniques applied to these challenges. A comprehensive review of the applications of DL techniques is then conducted from the perspective of analysis, forecast, management, and presented observation on each application. Critical points of DL models for improving performance are further discussed. In conclusion, several pressing problems of DL in load data analytics are identified, such as the accuracy gap between the actual and the expected, the generalization of hyperparameter setting, and the interpretation mechanism of DL output, which need special attention.