The worldwide aspiration for a sustainable energy future has led to an increasing deployment of variable and intermittent renewable energy sources (RESs). As a result, predicting and planning the operation of power grids has become more complex. Batteries can play a critical role to this problem as they can absorb the uncertainties introduced by RESs. In this paper, we solve the problem of computing a dispatch plan for a distribution grid with RESs and batteries with a novel approach based on Reinforcement Learning (RL). Although RL is not inherently suited for planning problems that require open loop policies, we have developed an iterative algorithm that calls a trained RL agent at each iteration to compute the dispatch plan. Since the feedback given to the RL agent cannot be directly observed because the dispatch plan is computed ahead of operation, it is estimated. Compared to the conventional approach of scenario-based optimization, our RL-based approach can exploit significantly more prior information on the uncertainty and computes dispatch plans faster. Our evaluation and comparative results demonstrate the accuracy of the computed dispatch plans as well as the adaptability of our agent to input data that diverge from the training data.