Despite the advancement of technical tools for the analysis of complex systems, the most important issue in solving water resource problems focuses on the interaction of human and natural systems. Agent-Based Model has been used as an effective tool for the development of integrated human and environmental models. One of the main challenges of this method is identifying and describing the main agents. In this study, three main approach including Genetic Algorithm, cooperative game theory and Agent-Based Model have been used to optimize water allocation in Tajan catchment. The proposed Agent-Based Model is a new equation for calculating stakeholder utility and simulating their interactions that can create a hydrological-environmental-human relationship for demand management and optimal water allocation. The results showed that the total benefit of cooperative game theory and Agent-Based Model relative to Genetic Algorithm has been increased 24 and 21% respectively. Although the total benefit in game theory is greater than the Agent-Based Model, but the Agent-Based Model considering the agents feedback propose a more comprehensive approach to optimal water allocation.