Crumb rubber (CR) concrete (CRC) characterized with high ductility and environment friendliness, has recently been studied widely in the field of green concrete. Generally, high volume fraction of CR in cementitious system leads to low strength. However, the effect of CR particle size has not been concluded yet. This paper focuses on the impact of CR size on the mechanical and microstructure properties of CRC. Two novel substitution approaches of CR for natural fine aggregate are adopted. In total, 10 groups are designed and experimentally tested for their workability in fresh state and mechanical properties in hardened state. Brittleness coefficient, fineness modulus, and micro‐morphology of CRC were analyzed and discussed. Results show that as size of CR decreases, workability improves significantly, apparent density and strength exhibit a firstly decrease and then increase trend, with the lowest value obtained around 1.18 mm CR size. The brittleness of concrete was significantly improved in the presence of CR, and the improvement was more significant for smaller‐sized CR. Weak ITZs between CR particles and interfacial porosity are the main reasons for the sharp strength deterioration. Crack arresting effect of CR makes the crack propagation path more tortuous, and smaller particle‐sized CR exhibits greater capability.