Surface defect identification plays a vital role in defective component rapid screening tasks in optics-related industries. However, the weakness and complexity of optical surface defects pose considerable challenges to their effective identification. To this end, a deep network based on multi-scale mixed kernels and structural re-parameterization is proposed to identify four manufacturing and two non-manufacturing optical surface defects. First, we design a multi-size mixed convolutional kernel with multiple receptive fields to extract rich shallow features for characterizing the defects with varying scales and irregular shapes. Then, we design an asymmetric mixed kernel integrating square, horizontal, vertical, and point convolutions to capture rotationally robust middle-and-deep features. Moreover, a structural re-parameterization strategy is introduced to equivalently convert the multi-branch architecture in the training phase into a deploy-friendly single-branch architecture in the inference phase, so that the model can obtain higher inference speed without losing any performance. Experiments on an optical surface defect dataset demonstrate that the proposed method is efficient and effective. It achieves a remarkable accuracy of 97.39% and an ultra-fast inference speed of 201.76 frames/second with only 5.23M parameters. Such a favorable accuracy–speed trade-off is capable of meeting the requirements of real-world optical surface defect identification applications.