Cracking is very common problem in cement mortar. Many past research has explored the prospect of using crumb rubber (CR) to overcome this issue. Different sizes of CR have been tested to measure its effect on the pore structure and mechanical strengths of cement mortar. Hence, this study has further modified the crumb rubber mortar mix by adding silica fume and synthetic fiber to improve its mechanical properties. The experimental results suggested that the optimum silica fume replacement of cement content was 5%. Hence, for the subsequent experiment with a fixed 5%silica fume replacement, the highest compressive strength of 26 MPa was achieved with 5% crumb rubber replacement. Finally, additional 0.1% of synthetic fiber added the modified crumb rubber mix to reduce the mix brittleness has produced a desirable compressive strength close to the control specimen which was significantly higher than the minimum threshold required by the standard. However, the water content ratio for the modified mortar mix should be further investigated as the present modified crumb rubber mix has lower workability. It is envisaged that the modified crumb rubber mortar mix has a sound potential to mitigate shrinkage cracking in cement mortar.