Digital respiratory sounds provide valuable information for telemedicine and smart diagnosis in an non-invasive way of pathological detection. As the typical continuous abnormal respiratory sound, wheeze is clinically correlated with asthma or chronic obstructive lung diseases. Meanwhile, the discontinuous adventitious crackle is clinically correlated with pneumonia, bronchitis, and so on. The detection and classification of both attract many studies for decades. However, due to the contained artifacts and constrained feature extraction methods, the reliability and accuracy of the classification of wheeze, crackle, and normal sounds need significant improvement. In this paper, we propose a novel method for the identification of wheeze, crackle, and normal sounds using the optimized S-transform (OST) and deep residual networks (ResNets). First, the raw respiratory sound is processed by the proposed OST. Then, the spectrogram of OST is rescaled for the Resnet. After the feature learning and classification are fulfilled by the ResNet, the classes of respiratory sounds are recognized. Because the proposed OST highlights the features of wheeze, crackle, and respiratory sounds, and the deep residual learning generates discriminative features for better recognition, this proposed method provides reliable access for respiratory disease-related telemedicine and E-health diagnosis. The experimental results show that the proposed OST and ResNet is excellent for the multi-classification of respiratory sounds with the accuracy, sensitivity, and specificity up to 98.79%, 96.27%, and 100%, respectively. The comparison results of the triple-classification of respiratory sounds indicate that the proposed method outperforms the deep-learning-based ensembling convolutional neural network (CNN) by 3.23% and the empirical mode decomposition-based artificial neural network (ANN) by 4.63%, respectively. INDEX TERMS Deep residual networks (ResNet), optimized S-transform (OST), respiratory sounds classification, crackle and wheeze detection.