The exact removal of material in abrasive belt grinding determines the final machining quality of the workpiece. However, it is difficult to determine the removal state of materials in actual processing, which is affected by factors such as abrasive belt wear and processing errors. Therefore, a multi-scale attention convolutional neural network for material removal state prediction method is proposed based on the analysis of displacement data. First, the first-order difference and sliding-window expansion methods for displacement data are adopted, making it possible to use displacement data for deep learning, which is the premise of material removal state prediction. Then, the multi-scale convolutional neural network is Employed to extract important features of the displacement data. Due to the different importance of different features, Squeeze-and-Excitation Networks are used to independently assign the importance of features based on the loss function, so that the model pays more attention to those main features and ignores the secondary features, which can improve the convergence speed and prediction accuracy of the model. The K6 cross-validation of experiment results shows that this method can accurately predict the material removal state with an average prediction accuracy of 87.9%, which can be practically applied to the online prediction of the material removal state in industrial processing to further control the processing quality.