To develop further understanding towards the role of a heterogeneous microstructure on tensile crack initiation and failure behavior in chopped carbon fiber chip‐reinforced composites, uni‐axial tensile tests are performed on coupons cut from compression molded plaque with varying directions. Experimental results indicate that failure initiation is relevant to the strain localization, and a new criterion with the nominal modulus to predict the failure location is proposed based on the strain analysis. Furthermore, optical microscopic images show that the nominal modulus is determined by the chip orientation distribution. At the area with low nominal modulus, it is found that chips are mostly aligning along directions transverse to loading direction and/or less concentrated, while at the area with high nominal modulus, more chips are aligning to tensile direction. On the basis of failure mechanism analysis, it is concluded that transversely oriented chips or resin‐rich regions are easier for damage initiation, while longitudinally oriented chips postpone the fracture. Good agreement is found among failure mechanism, strain localization and chip orientation distribution. POLYM. COMPOS., 40:E962–E974, 2019. © 2018 Society of Plastics Engineers