Over the last decades, researchers have studied the Multi-Objective Optimization problem (MOO) for Multi-Agent Systems (MASs). However, most of them consider the problem formulation to be an unprioritized sum of objective functions, and no work has reviewed problems with the formulation of the prioritized sum of objective functions to facilitate the study of the subject and identify the needs arising from it. In the context of Multi-Robot Systems (MRSs), most studies only focus on the mathematical development of their proposed MOO algorithm without paying attention to the application. In practice, there is not a comprehensive review to identify the reliable algorithms already applied to real platforms. Using a mapping and state-of-the-art review, this paper aims to fill these gaps by first offering a detailed overview of the discrete-time MOO methods for MASs. More specifically, we classify existing MOO methods based on the formulation of the problem into the sum of objective functions and the prioritized sum of objective functions. Secondly, we review the applications of these methods in MRSs and the practical implementation of MOO algorithms on real MRS. Finally, we suggest future research directions to extend the existing methods to more realistic approaches, including open problems in the new research area of the prioritized sum of objective functions and practical challenges for the implementation of the existing methods in robotics. This work introduces the field of MAS to researchers and enables them to position themselves in the current research trends.INDEX TERMS Multi-agent systems, multi-objective optimization, multi-robot systems, prioritized sum of objective functions.