During the long-term service of slab track, various external factors (such as complicated temperature) can result in a series of slab damages. Among them, slab arching changes the structural mechanical properties, deteriorates the track geometry conditions, and even threatens the operation of trains. Therefore, it is necessary to detect slab arching accurately to achieve effective maintenance. However, the current damage detection methods cannot satisfy high accuracy and low cost simultaneously, making it difficult to achieve large-scale and efficient arching detection. To this end, this paper proposed a vision-based arching detection method using track geometry data. The main works include: (1) data nonlinear deviation correction and arching characteristics analysis; (2) data conversion and augmentation; (3) design and experiments of convolutional neural network- based detection model. The results show that the proposed method can detect arching damages effectively, and the F1-score reaches 98.4%. By balancing the sample size of each pattern, the performance can be further improved. Moreover, the method outperforms the plain deep learning network. In practice, the proposed method can be employed to detect slab arching and help to make maintenance plans. The method can also be applied to the data-based detection of other structural damages and has broad prospects.