The early detection of tumor is necessary to save a number of lives. In women, the temperature of the affected area of the tumor is warmer than the unaffected area; therefore the thermography technique can be used to capture the cancerous breast images with a thermal infrared by identifying the temperature difference between them. Color enhancement of the captured breast image is an important consideration for early detection of tumor at this stage. Therefore, in this paper, we propose a self‐adaptive gray level histogram equalization approach to enhance the color of the IR image for early detection of the tumor. This approach does not require any manual feeding of parameters toward images. The final classification of tumorous and non‐tumorous breast images can obtain through certain procedures, which includes, image acquisition, pre‐processing, segmentation, feature extraction and classification. This paper emphasizes the support vector machine (SVM) technique to classify the tumor IR thermography images. The proposed approach is implemented in MATLAB and the experimental results shows an outstanding color enhancement of IR images and better classification compared to other existing methods such as CLAHE, BIi‐histogram equalization and adaptive histogram equalization. The performance was evaluated by using evaluation metrics such as sensitivity, accuracy, and specificity of thermography breast image by the SVM classifier adapted with various color enhancement approaches are found to be 91.6%, 90%, and 87.5%. This approach helps in medical field for early diagnosis with high reliability.