Voluntary participation, as an effective mechanism to promote cooperation in game theory, has been widely concerned. In the meanwhile, reciprocal rewarding plays an important role in motivating individual initiative. Inspired by this phenomenon, we investigate the effect of reciprocal rewarding on the evolutionary cooperation in spatial social dilemmas, including prisoner's dilemma game and the snowdrift game with voluntary participation. In our model, a cooperative individual fitness will be redefined if one could obtain additional incentive bonus which is proportional to the number of cooperative neighbors. Moreover, each individual is a pure strategist in the spatial structured population and could only choose one of three strategies-cooperation, defection and being a loner. Through numerical simulations, we have confirmed that, compared with the traditional situation, reciprocal rewarding and the payoff of loner can significantly promote the cooperative behavior among the population, and the greater the contribution of reciprocal rewarding/payoff of loner, the more obvious the promoting effect on cooperation. In addition, we also find that there is a condition for loner to make the system fall into the three-strategy cyclic dominance, that is, the payoff of loner can not be too small or too large, which will destroy the situation of cyclic dominance. With regard to these results, it is strongly unveiled that reciprocal rewarding has a positive role to resolve the social dilemmas in the evolution of cooperation.