Breast cancer is one of the most common women cancers in the world. In this paper, a new approach based on thermography for the early detection of breast abnormality is proposed. The study involved 80 breast thermograms collected from the PROENG public database which consists of 50 healthy breasts and 30 with some findings. Image processing techniques such as segmentation, texture analysis and mathematical morphology were used to train a support vector machine (SVM) classifier for automatic detection of breast abnormality. After conducting several tests, we obtained very interesting and motivating results. Indeed, our method  showed a high performance in terms of sensitivity of 93.3%, a specificity of 90% and an accuracy of 91.25%. The final results let us conclude that infrared thermography with the help of an adequate automatic classification algorithm can be a valuable and reliable complementary tool for radiologist in detecting breast cancer and thereby helping to reduce mortality rates.