Using ab initio evolutionary simulations, we have explored the potential crystal structures with up to 18 atoms in the unit cell for all possible stoichiometries of the Zr-B system. In addition to the reported ZrB, ZrB2, ZrB12, oP8-ZrB and Zr3B4, two extraordinary Zr2B3 and Zr3B2 have been found to be both mechanically and dynamically stable, as verified by the calculated elastic constants and phonon dispersions. The calculated formation enthalpies reveal that both the new phases may be synthesized and found in experiments. It also reveals that pressure is beneficial for the synthesis of all available phases, except for the ZrB phase. In addition, the values of the Vickers hardness for five Zr-B compounds are predicted utilizing two different models. Contrary to the known hard phases of ZrB2 and ZrB12, the two new compounds both have low values of hardness (less than 10 GPa), which is consistent with their excellent ductility deduced from the bulk and shear moduli. Electronic structure calculations suggest that the new phases are both metallic, while electronic density maps show strong ionic bonding characteristics between the Zr and B atoms. The crystal orbital Hamilton population (COHP) diagrams have also been calculated in order to further analyze the bonding features of the uncovered two novel phases.