Developing highly active nonprecious electrocatalysts with superior durability for both the hydrogen evolution reaction (HER) and oxygen evolution reaction (OER) is crucial to improve the efficiency of overall water splitting but remains challenging. Here, a novel superhydrophilic Co4N‐CeO2 hybrid nanosheet array is synthesized on a graphite plate (Co4N‐CeO2/GP) by an anion intercalation enhanced electrodeposition method, followed by high‐temperature nitridation. Doping CeO2 into Co4N can favor dissociation of H2O and adsorption of hydrogen, reduce the energy barrier of intermediate reactions of OER, and improve the compositional stability, thereby dramatically boosting the HER performance while simultaneously inducing enhanced OER activity. Furthermore, the superhydrophilic self‐supported electrode with Co4N‐CeO2 in situ grown on the conductive substrate expedites electron conduction between substrate and catalyst, promotes the bubble release from electrode timely and impedes catalyst shedding, ensuring a high efficiency and stable working state. Consequently, the Co4N‐CeO2/GP electrode shows exceptionally low overpotentials of 24 and 239 mV at 10 mA cm−2 for HER and OER, respectively. An alkaline electrolyzer by using Co4N‐CeO2/GP as both the cathode and anode requires a cell voltage of 1.507 V to drive 10 mA cm−2, outperforming the Pt/C||RuO2 electrolyzer (1.540 V@10 mA cm−2). More significantly, the electrolyzer has extraordinary long‐term durability at a large current density of 500 mA cm−2 for 50 h, revealing its potential in large‐scale applications.