In the restructured environment of the power industry, various commodities such as energy and operating reserves may be provided through simultaneous auctions. Prediction of market players' behavior in the auctions and simulation of the markets' environment can assist market decision-makers in evaluating specific policies before enforcing them in the real environment. Considering effects of the energy and varieties of reserve markets and also their interactions in the simulations is of high importance, which leads to more realistic simulation results. In this paper, an approach based on a multiagent system is proposed for simulating the simultaneous energy, spinning reserve, and replacement reserve market auctions, in which bidding of each agent is carried out based on a reinforcement learning algorithm. The proposed method is applied on a sample system, through which the impacts of considering lost opportunity cost in the payment model are examined.