Solid-state refrigeration technology is expected to replace conventional gas compression refrigeration technology because it is environmentally friendly and highly efficient. Among various solid-state magnetocaloric materials, Ni-Mn-based ferromagnetic shape memory alloys (SMAs) have attracted widespread attention due to their multifunctional properties, such as their magnetocaloric effect, elastocaloric effect, barocaloric effect, magnetoresistance, magnetic field-induced strain, etc. Recently, a series of in-depth studies on the thermal effects of Ni-Mn-based magnetic SMAs have been carried out, and numerous research results have been obtained. It has been found that poor toughness and cyclic stability greatly limit the practical application of magnetic SMAs in solid-state refrigeration. In this review, the influences of element doping, microstructure design, and the size effect on the strength and toughness of Ni-Mn-based ferromagnetic SMAs and their underlying mechanisms are systematically summarized. The pros and cons of different methods in enhancing the toughness of Ni-Mn-based SMAs are compared, and the unresolved issues are analyzed. The main research directions of Ni-Mn-based ferromagnetic SMAs are proposed and discussed, which are of scientific and technological significance and could promote the application of Ni-Mn-based ferromagnetic SMAs in various fields.