Inorganic nanoparticles have been widely investigated as therapeutic agents for cancer treatments in biomedical fields due to their unique physical/chemical properties, versatile synthetic strategies, easy surface functionalization and excellent biocompatibility. This review focuses on the discussion of several types of inorganic nanoparticle-based cancer therapeutic agents, including gold nanoparticles, magnetic nanoparticles, upconversion nanoparticles and mesoporous silica nanoparticles. Several cancer therapy techniques are briefly introduced at the beginning. Emphasis is placed on how these inorganic nanoparticles can provide enhanced therapeutic efficacy in cancer treatment through site-specific accumulation, targeted drug delivery and stimulated drug release, with elaborations on several examples to highlight the respective strategies adopted. Finally, a brief summary and future challenges are included.