The present review aims to analyze the application of infrared thermal imaging, aided by bio-heat models, as a tool for the diagnosis of skin and breast cancers. The state of the art of the related technical procedures, bio-heat transfer modeling, and thermogram post-processing methods is comprehensively reviewed. Once the thermal signatures of different malignant diseases are described, the updated thermographic techniques (steady-state and dynamic) used for cancer diagnosis are discussed in detail, along with the recommended best practices to ensure the most significant thermal contrast observable between the cancerous and healthy tissues. Regarding the dynamic techniques, particular emphasis is placed on innovative methods, such as lock-in thermography, thermal wave imaging, and rotational breast thermography. Forward and inverse modeling techniques for the bio-heat transfer in skin and breast tissues, supporting the thermographic examination and providing accurate data for training artificial intelligence (AI) algorithms, are reported with a special focus on real breast geometry-based 3D models. In terms of inverse techniques, different data processing algorithms to retrieve thermophysical parameters and growth features of tumor lesions are mentioned. Post-processing of infrared images is also described, citing both conventional processing procedures and applications of AI algorithms for tumor detection.