Optimizing binders by using industrial waste and then adjusting microstructure has potential to develop low production cost and low carbon emission ultra-high performance concrete (UHPC) to meet different practical engineering requirements. This study first investigates the influence of silica fume, cenosphere and fly ash on paste mixing time, flowability, flexural and compressive strength of UHPC by using three-factor and four-level orthogonal test. Moreover, the effect of fly ash and slag content were further discussed, and the modification mechanisms of binders were revealed by analyzing pore structure, interface and characteristics of hydration products. The results show that silica fume and cenosphere are the most sensitive factors affecting paste mixing time and flowability of UHPC, respectively. The sensitivity of binders on compressive and flexural strength of UHPC can be ranked according to silica fume > fly ash > cenosphere and cenosphere > fly ash > silica fume, respectively. The binders’ composition for low-carbon UHPC with compressive strength grade of 120 MPa is that cement: cenosphere: silica fume: fly ash equals to 1:0.33:0.33:56, and the total amount of fly ash reaches 40%. Meanwhile, the 28d compressive/flexural strength decreases and the flowability increases with the increase of fly ash content, resulting from pore volume with size of 5-50 nm and Ca/Si ratio on the interface between steel fiber and matrix increase. The flowability of UHPC with slag is reduced and the cumulative pore volume with size of 50 nm-5 um pores is increased due to the influence of slag morphology, but the porosity and interfacial Ca/Si ratio is reduced because of the high pozzolanic activity of slag, leading to similar compressive and flexural strength with that of UHPC with fly ash.