Magnetic cooling, a solid-state refrigeration technology based on the magnetocaloric effect (MCE), has attracted significant attention in space cooling due to its high energy-efficiency and environmental friendliness. Transition metal-based magnetocaloric materials (MCMs) with the merit of low-cost have emerged as promising candidates for efficient magnetic refrigeration applications. This review explores the intricate relationship between microstructure and multiple properties (e.g. magnetocaloric properties, mechanical stability, thermal conductivity, and functional reversibility) of these materials. A variety of microstructural manipulation approaches (e.g. crystallographic texture, precipitates, micropores, atomic-scale defects, size effect, and composites) are examined for their effects on the comprehensive performance of MCMs. We show that the microstructure design provides an effective tool to achieve excellent performance in multiple aspects, which may facilitate the commercialization of transition-metal based magnetocaloric materials.