Many asteroids are likely to be have been shattered by collisions into fragments and reaccumulated as gravitationally‐bound rubble piles. These bodies may contain large porosities, although this picture may be complicated by compaction inside the asteroid body. Estimates of asteroid mass and volume imply a negative correlation between size and porosity for stony asteroids. Asteroids that are suspected to be metallic appear to contain larger porosities than stony asteroids of similar sizes. To understand these observations, we develop models for cold compaction of fragments of different materials. The initial boulder size distributions are assumed to be narrow. We focus on macro‐porosity between the boulders and do not consider micro‐porosity inside the boulders. In our model of silicate/chondritic boulders, compaction is assumed to occur through cataclastic fracturing, which creates small pieces that fill the pores between residual large boulders, leading to fractal‐like distributions. This fracturing occurs when compression leads to stresses exceeding the tensile strength of the boulders. Combining this model with data on meteorite strength, we suggest that the compaction of chondritic boulders can be significant at pressures of several megapascals. In our model of metal boulders, we consider cold welding and boulder deformation (through ductile yielding or brittle‐like fracturing, depending on the stress and intrinsic crack size). Given the properties of iron meteorites, we infer that compaction in metallic rubble piles, caused by ductile or brittle deformation, is small, and that cold welding may lead to large (≳50%) porosities if the boulders are of ∼1 m sizes.