It has long been recognized that exposure to irradiation environments could dramatically degrade the mechanical properties of nuclear structural materials, i.e., irradiation-hardening and embrittlement. With the development of numerical simulation capability and advanced experimental equipment, the mysterious veil covering the fundamental mechanisms of irradiation-hardening and embrittlement has been gradually unveiled in recent years. This review intends to offer an overview of the fundamental mechanisms in this field at moderate irradiation conditions. After a general introduction of the phenomena of irradiation-hardening and embrittlement, the formation of irradiation-induced defects is discussed, covering the influence of both irradiation conditions and material properties. Then, the dislocation-defect interaction is addressed, which summarizes the interaction process and strength for various defect types and testing conditions. Moreover, the evolution mechanisms of defects and dislocations are focused on, involving the annihilation of irradiation defects, formation of defect-free channels, and generation of microvoids and cracks. Finally, this review closes with the current comprehension of irradiation-hardening and embrittlement, and aims to help design next-generation irradiation-resistant materials.