Applications in wireless sensor networks (WSNs) are rapidly spreading out over the world. The one critical point of WSNs is energy consumption, where the transmitted data is limited by battery energy. Solar energy is used to handle the depletion of the battery energy via photo voltaic (PV) panels. A solar energy harvesting WSN (SEH-WSN) node utilizes exponential decision-dynamic duty cycle scheduling based on prospective increase in energy (ED-DSP) to save battery energy by adjusting the duty cycle from an exponential curve and future solar energy. To estimate the prospective solar energy, a prediction technique is applied, but does not guarantee 100% accuracy. Hence, this paper proposes a Markov Decision Process (MDP) to schedule a duty cycle of an SEH-WSN node instead of the ED-DSP depending on the predicted energy. We evaluate its performance via MATLAB simulations with simple irradiance models and real annual irradiance data. The results show that the MDP policy outperforms the ED-DSP.