Electrochemical water splitting is an environmentally friendly and effective energy storage method. However, it is still a huge challenge to prepare non‐noble metal based electrocatalysts that possess high activity and long‐term durability to realize efficient water splitting. Here, we present a novel method of low‐temperature phosphating for preparing CoP/Co3O4 heterojunction nanowires catalyst on titanium mesh (TM) substrate that can be used for oxygen evolution reaction (OER), hydrogen evolution reaction (HER), and overall water splitting. CoP/Co3O4@TM heterojunction showed an excellent catalytic performance and long‐term durability in 1.0 M KOH electrolyte. The overpotential of CoP/Co3O4@TM heterojunction was only 257 mV at 20 mA cm−2 during the OER process, and it could work stably more than 40 h at 1.52 V versus reversible hydrogen electrode (vs. RHE). During the HER process, the overpotential of CoP/Co3O4@TM heterojunction was only 98 mV at −10 mA cm−2. More importantly, when used as anodic and cathodic electrocatalyst, they achieved 10 mA cm−2 at 1.59 V. The Faradaic efficiencies of OER and HER were 98.4 % and 99.4 %, respectively, outperforming Ru/Ir‐based noble metal electrocatalysts and other non‐noble metal electrocatalysts for overall water splitting.