Directed Energy Deposition (DED), a branch of AM processes, has emerged as a significant technique for fabricating large metal components in sectors such as aerospace, automotive, and healthcare. DED is characterized by its high deposition rate and scalability, which stand out among other AM processes. However, it encounters critical issues such as residual stresses, distortion, porosity, and rough surfaces resulting from rapid melting and solidification. As a novel advancement, Ultrasonic-Assisted Directed Energy Deposition (UA-DED) integrates ultrasonic oscillations into DED aimed at addressing these challenges. Herein, the latest research related to the UA-DED process and the current major challenges of the DED process, residual stresses, porosity, and crack defects are critically reviewed. Subsequently, the paper also details the working principle and system components of UA-DED technology and reviews the material improvement by introducing UA into the DED process, grain, porosity, tensile properties, and deposition defects. The most critical optimization methods of process parameter variables for UA and the different material interaction mechanisms between UA and DED processes are identified and discussed in detail. Finally, the perspectives on the research gap and potential future developments in UA-DED are also discussed.