Impact location detection plays an important role in the structural health monitoring of metal materials. However, the methods of metal material impact location detection based on physical analysis are often limited by the extraction accuracy of some parameters such as material and structure parameters and time difference calculation. Therefore, this paper develops a deep residual network method for impact location detection, time-frequency characteristic deep residual network (TF-DRN). This method takes the four-channel short-time Fourier transform (STFT) time-frequency graph as input, uses the unique residual network architecture to automatically extract the advanced features, and then uses the global average pooling layer and the full connection layer to establish the mapping between the advanced features and the impact location, so as to detect the collision location. By introducing regularization and batch normalization, the problems of gradient disappearance and gradient explosion are alleviated, and the generalization and efficiency of impact location detection are further improved. The experimental results show that on an 800mm×800mm×2.5mm aluminum plate, the average error of the validation set and the test set are 0.85cm and 1.33cm respectively, and the performance of the method is significantly better than that of CNN, ResNet18 and ResNet33 networks.