Given the burgeoning demand for construction of structures and high-rise buildings, controlling the structural vibrations under earthquake and other external dynamic forces seems more important than ever. Vibration control devices can be classified into passive, active and hybrid control systems. The technologies commonly adopted to control vibration, reduce damage, and generally improve the structural performance, include, but not limited to, damping, vibration isolation, control of excitation forces, vibration absorber. Tuned Mass Dampers (TMDs) have become a popular tool for protecting structures from unpredictable vibrations because of their relatively simple principles, their relatively easy performance optimization as shown in numerous recent successful applications. This paper presents a critical review of active, passive, semi-active and hybrid control systems of TMD used for preserving structures against forces induced by earthquake or wind, and provides a comparison of their efficiency, and comparative advantages and disadvantages. Despite the importance and recent advancement in this field, previous review studies have only focused on either passive or active TMDs. Hence this review covers the theoretical background of all types of TMDs and discusses the structural, analytical, practical differences and the economic aspects of their application in structural control. Moreover, this study identifies and highlights a range of knowledge gaps in the existing studies within this area of research. Among these research gaps, we identified that the current practices in determining the principle natural frequency of TMDs needs improvement. Furthermore, there is an increasing need for more complex methods of analysis for both TMD and structures that consider their nonlinear behavior as this can significantly improve the prediction of structural response and in turn, the optimization of TMDs.