Solar light harvesting by photocatalytic H evolution from water could solve the problem of greenhouse gas emission from fossil fuels with alternative clean energy. However, the development of more efficient and robust catalytic systems remains a great challenge for the technological use on a large scale. Here we report the synthesis of a sol-gel prepared mesoporous graphitic carbon nitride (sg-CN) combined with nickel phosphide (Ni P) which acts as a superior co-catalyst for efficient photocatalytic H evolution by visible light. This integrated system shows a much higher catalytic activity than the physical mixture of Ni P and sg-CN or metallic nickel on sg-CN under similar conditions. Time-resolved photoluminescence and electron paramagnetic resonance (EPR) spectroscopic studies revealed that the enhanced carrier transfer at the Ni P-sg-CN heterojunction is the prime source for improved activity.