Unavoidable variations in size and position of the building blocks of photonic crystals cause light scattering and extinction of coherent beams. We present a new model for both 2 and 3-dimensional photonic crystals that relates the extinction length to the magnitude of the variations.The predicted lengths agree well with our new experiments on high-quality opals and inverse opals, and with literature data analyzed by us. As a result, control over photons is limited to distances up to 50 lattice parameters (∼ 15 µm) in state-of-the-art structures, thereby impeding large-scale applications such as integrated circuits. Conversely, scattering in photonic crystals may lead to novel physics such as Anderson localization and non-classical diffusion.