Graph neural networks have been widely used in the field of bearing fault diagnosis, which can deal with non-Euclidean space data and dig deep the relationship between signals. However, most graph neural networks do not distinguish the importance of nodes in information aggregation, and do not take edge noise and data redundancy into account when constructing the graph structure, which affects the diagnostic accuracy. To solve these problems, a fault diagnosis method of graph attention network based on sparsity structure pruning is proposed. Firstly, a sparsity coefficient is introduced to construct the graph structure, and pruning operations are carried out according to the coefficient and the weight of the edges to avoid invalid fusion of information. Then, a graph attention network model based on sparsity structure pruning is constructed, and features of different scales are aggregated into new node representations through multi-head attention mechanism. Finally, the fault diagnosis of bearing is carried out according to the extracted signal discrimination characteristics. To verify the effectiveness of the proposed method, experiments are performed on two different fault diagnosis datasets and compared with other graph neural network methods. The results show that the accuracy and stability of the proposed method are superior to other methods even under the condition of low signal to noise ratio (SNR).