The purpose of this paper is to propose a two-step optimization method to design tuned inertial mass dampers for elastic-plastic MDOF structures using the time-frequency dual domain performance evaluation. Although the frequency-domain-based approach allows us to obtain the optimal design independent of the characteristics of disturbances, it cannot consider the elastic-plastic response of structures. On the other hand, the timedomain design is highly dependent on the characteristics of design earthquake ground motions. The proposed two-step method resolves these issues by use of dual-domain information. Finally, the response analysis for recorded ground motions demonstrates the effectiveness of the proposed design method.