Undispersed filler agglomerates or other substantial inclusions/contaminants in rubber can act as large crack precursors that reduce the strength and fatigue lifetime of the material. To demonstrate this, we use tensile strength (stress at break, σb) data from 50 specimens to characterize the failure distribution behavior of carbon black (CB) reinforced styrene-butadiene rubber (SBR) compounds. Poor mixing was simulated by adding a portion of the CB late in the mixing process, and glass beads (microspheres) with 517 μm average diameter were introduced during milling to reproduce the effects of large inclusions. The σb distribution was well described with a simple unimodal Weibull distribution for the control compound, but the tensile strengths of the poor CB dispersion material and the compounds with the glass beads required bimodal Weibull distributions. For the material with the lowest level of glass beads—corresponding to less than one microsphere per test specimen—the bimodal failure distribution spanned a very large range of σb from 13.7 to 22.7 MPa in contrast to the relatively narrow σb distribution for the control from 18.4 to 23.8 MPa. Crack precursor size (c0) distributions were also inferred from the data, and the glass beads introduced c0 values in the 400 μm range compared to about 180 μm for the control. In contrast to σb, critical tearing energy (tear strength) was unaffected by the presence of the CB agglomerates and glass beads, because the strain energy focuses on the pre-cut macroscopic crack in the sample during tear testing rather than on the microscopic crack precursors within the rubber. The glass beads were not detected by conventional filler dispersion measurements using interferometric microscopy, indicating that tensile strength distribution characterization is an important complementary approach for identifying the presence of minor amounts of large inclusions in rubber.