Tuberculosis (TB) is an infectious disease caused by Mycobacterium. It primarily impacts the lungs but can also endanger other organs, such as the renal system, spine, and brain. When an infected individual sneezes, coughs, or speaks, the virus can spread through the air, which contributes to its high contagiousness. The goal is to enhance detection recognition with an X-ray image dataset. This paper proposed a novel approach, named the Tuberculosis Segmentation-Guided Diagnosis Model (TSSG-CNN) for Detecting Tuberculosis, using a combined semantic segmentation and adaptive convolutional neural network (CNN) architecture. The proposed approach is distinguished from most of the previously proposed approaches in that it uses the combination of a deep learning segmentation model with a follow-up classification model based on CNN layers to segment chest X-ray images more precisely as well as to improve the diagnosis of TB. It contrasts with other approaches like ILCM, which is optimized for sequential learning, and explainable AI approaches, which focus on explanations. Moreover, our model is beneficial for the simplified procedure of feature optimization from the perspectives of approach using the Mayfly Algorithm (MA). Other models, including simple CNN, Batch Normalized CNN (BN-CNN), and Dense CNN (DCNN), are also evaluated on this dataset to evaluate the effectiveness of the proposed approach. The performance of the TSSG-CNN model outperformed all the models with an impressive accuracy of 98.75% and an F1 score of 98.70%. The evaluation findings demonstrate how well the deep learning segmentation model works and the potential for further research. The results suggest that this is the most accurate strategy and highlight the potential of the TSSG-CNN Model as a useful technique for precise and early diagnosis of TB.