It is crucial to realize the municipal solid waste (MSW) classification in terms of its treatments and disposals. Deep learning used for the classification of residual waste and wet waste from MSW was considered as a promising method. While few studies reported using the method of deep learning with transfer learning to classify organic waste and residual waste. Thus, this study aims to discuss the effect of the transfer learning on the performance of different deep learning structures, VGGNet-16 and ResNet-50, for the classification of organic waste and residual waste, which were compared in terms of the training time, confusion matric, accuracy, precision, and recall. In addition, the algorithms of PCA and t-SNE were also adopted to compare the representation extracted from the last layer of various deep learning models. Results indicated that transfer learning could shorten the training time and the training time of various deep learning follows this order: VGGNet-16 (402 s) > VGGNet-16 with TL (272 s) > ResNet-50 (238 s) > ResNet-50 with TL (223 s). Compared with the method of PAC, waste representations were better separated from high dimension to low dimension by t-SNE. The values of organic waste in terms of F1 score follows this order: ResNet-50 with transfer learning (97.8%) > VGGNet-16 with transfer learning (97.1%) > VGGNet-16 (95.0%) > ResNet-50 (92.5%).Therefore, the best performance for the classification of organic and residual waste was ResNet-50 with transfer learning, followed by VGGNet-16 with transfer learning and VGGNet-16, and ResNet-50 in terms of accuracy, precision, recall, and F1 score.