Multi-agent systems and agent-oriented methodologies support analysis, characterization and development of complex software systems. These methodologies introduce different definitions for the essential components of multi-agent systems and cover different phases of the system development life cycle. Therefore, appropriate frameworks for evaluation and comparison of different methodologies would support developers to adopt the best methodology, or a combination of different methodologies, based on the project requirements. This review covers the system development phases and the main conceptual components in the context of multi-agent systems. Then, the evaluation frameworks proposed in the literature for comparison and evaluation of agent-oriented methodologies are reviewed. Evaluation frameworks proposed in the literature are categorized into three categories: methodology-based, phase-based and feature-based evaluation frameworks. The paper concludes with the agent-oriented methodologies’ usage challenges, their current limitations and potential future directions.