In recent years, interdisciplinary research has become increasingly popular within the scientific community. The fields of materials science and chemistry have also gradually begun to apply the machine learning technology developed by scientists from computer science. Graph neural networks (GNNs) are new machine learning models with powerful feature extraction, relationship inference, and compositional generalization capabilities. These advantages drive researchers to design computational models to accelerate material property prediction and new materials design, dramatically reducing the cost of traditional experimental methods. This review focuses on the principles and applications of the GNNs. The basic concepts and advantages of the GNNs are first introduced and compared to the traditional machine learning and neural networks. Then, the principles and highlights of seven classic GNN models, namely crystal graph convolutional neural networks, iCGCNN, Orbital Graph Convolutional Neural Network, MatErials Graph Network, Global Attention mechanism with Graph Neural Network, Atomistic Line Graph Neural Network, and BonDNet are discussed. Their connections and differences are also summarized. Finally, insights and prospects are provided for the rapid development of GNNs in materials science at the atomic scale.