Gearbox fault diagnosis based on traditional deep learning often needs a large number of samples. However, the gearbox fault samples are limited in practical engineering, which could lead to poor diagnosis performance. Based on the above problems, this paper proposes a gearbox fault diagnosis method based on Gramian angular field (GAF) and TLCA-MobileNetV3 to achieve fast and accurate limited sample recognition under varying working conditions, and further achieve the cross-component fault diagnosis within the gearbox. First, the 1D signals are converted into 2D images through GAF. Second, a lightweight convolutional neural network is established. Coordinate attention (CA) is integrated into the network to establish remote dependency in space and improve the ability of feature extraction. The optimal strategy for model training is determined. Finally, a transfer learning strategy is designed. The lower structures of network are frozen. The higher structures of network are fine-tuned using limited samples. Through experimental verification, the proposed network could achieve limited sample fault diagnosis under varying working conditions and cross-component conditions.