Purpose Breast cancer is one of the deadliest cancers among women worldwide which its early detection may significantly reduce its mortality rate. Thermgraphy is a new, non-invasive, non-painful, and low-cost modality that detects abnormalities by detecting heat from the breast surface. Method Recent research has applied deep learning to early breast cancer diagnosis via thermography, using only the frontal view of thermograms. We combine several views of thermal images to improve the performance of pre-trained deep learning architectures in this article. This goal is achieved by combining frontal-45 data with lateral-45 and lateral45 thermograms to construct a detection model that utilizes transfer learning. Result Research in this area uses the Database for Mastology Research (DMR) with infrared images. In this study, transfer based deep learning methods are demonstrated to be effective in fusing several views of thermograms to diagnose breast cancer in a manner that can result in a sensitivity increase of 2-15 percent and a specificity increase of 2-30 percent compared to other deep learning-based or handcrafted schemes. Conclusion Using multiple views of thermograms and transfer learning, this paper proposes a method for improving breast cancer diagnosis. Using methods based on deep learning and methods based on hand-crafted features, we evaluated the performance of the proposed model. Using the obtained results as a basis for future research, the proposed design can be improved and developed as a valid approach in interpreting breast thermography images.