Tonsillopharyngitis is a sudden onset contagious infection of the pharynx and tonsils. Patients experience a rapid general condition and loss of workforce. In addition to affecting patients, it spreads and affects other individuals. In addition, it causes severe complications and increases hospital costs. Therefore, early and accurate diagnosis is essential. In this study, a hybrid model is developed for the diagnosis of tonsillopharyngitis. First, the heat maps of the images in the original data set by applying the Gradient-weighted Class Activation Mapping (Grad-Cam) method. In the proposed model, feature maps are obtained from the original and heatmap datasets using the Darknet53 architecture as the base. It is aimed to increase the performance of the proposed model by bringing together different features of the same image. After the feature map obtained after the feature fusion step is optimized with the Relief method, classification is carried out using an SVM shallow classifier.