Mitochondrial heat shock protein 90 (mtHsp90), including Tumor necrosis factor receptor-associated protein 1 (TRAP1) and Hsp90 translocated from cytoplasm, modulating cellular metabolism and signaling pathways by altering the conformation, activity, and stability of numerous client proteins, and is highly expressed in tumors. mtHsp90 inhibition results in the destabilization and eventual degradation of its client proteins, leading to interference with various tumor-related pathways and efficient control of cancer cell development. Among these compounds, gamitrinib, a specific mtHsp90 inhibitor, has demonstrated its safety and efficacy in several preclinical investigations and is currently undergoing evaluation in clinical trials. This review aims to provide a comprehensive overview of the present knowledge pertaining to mtHsp90, encompassing its structure and function. Moreover, our main emphasis is on the development of mtHsp90 inhibitors for various cancer therapies, to present a thorough overview of the recent pre-clinical and clinical advancements in this field.