Proton exchange membrane fuel cells (PEMFCs) are promising components in the renewable energy field due to their high energy efficiency and low pollution output. However, these cells are also characterized by considerable nonlinearity, which in turn adversely affects the PEMFC output voltage. Conventional control algorithms cannot guarantee sufficient output voltage control, as they lack the robust adaptive ability required for adapting to these fluctuations in PEMFCs. In this paper, an optimal output voltage controller based on distributed deep reinforcement learning, which controls the output voltage by regulating the fuel input of the PEMFC, is proposed. In addition, an ensemble intelligence exploration multi-delay deep deterministic policy gradient (EIM-DDPG) algorithm is proposed for this controller. An ensemble intelligence exploration policy plus a classification experience replay mechanism are included within the EIM-DDPG algorithm to improve the exploration ability of the algorithm and thus increase the robustness and adaptive capability of the controller. As a result, the model-free optimal output voltage controller offers high robustness and adaptability. The simulation results in this paper demonstrate that the proposed optimal controller can realize the effective control of PEMFC output voltage.This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.