Based on Gravity Recovery and Interior Laboratory (GRAIL) observations and the Lunar Orbiter Laser Altimeter (LOLA) crater database, we constrain the spatial variations in the Moon's crustal porosity (ϕc) to a depth of several kilometers using the gravity anomalies of 4,864 midsized craters (those with a diameter of 20–100 km). For each crater, we estimated the local ϕc by quantifying the gravitational effects of impact‐induced porosity change and postimpact breccia infill. The crustal porosity model was uniquely determined assuming a global mean ϕc of 12%, although a trade‐off exists between the porosity of postimpact breccia infill and the (preimpact) crustal porosity that results in zero net porosity change underlying the crater floor. At this crustal porosity the effects of impact bulking and compaction compensate each other to yield zero crater residual Bouguer anomaly (RBA), when the effect of postimpact infill is excluded. For lower crustal porosities, bulking dominates to produce a negative RBA; for higher crustal porosities, compaction dominates to produce a positive RBA. Spatial kriging and statistical tests suggest lower‐than‐average ϕc in the western nearside maria and southern South Pole‐Aitken (SP‐A) basin, in contrast to higher‐than‐average values in the southwestern periphery of the nearside maria. Most of the large impact basins, presumably formed before the majority of the midsized craters we analyzed, show reduced porosities relative to their immediate surroundings.