Synapse specificity is a basic feature of synaptic plasticity, but it remains unclear how synapse-specific signaling is achieved if postsynaptic membrane proteins can diffuse laterally between synapses. We monitored movements of AMPA receptors (AMPARs) on the surface of mature neurons to investigate the role of lateral diffusion in constitutive AMPAR trafficking and to assess the influence of membrane architecture on the surface distribution of synaptic proteins. Our data show that lateral diffusion is responsible for the continual exchange of a substantial pool of AMPARs at the spine surface. Furthermore, we found that a general characteristic of membrane proteins is that their movement into and out of spines is slow compared with that in nonspiny membrane. This shows that lateral diffusion is dependent on spine morphology and is restricted at the spine neck. These results demonstrate the importance of lateral diffusion in trafficking of AMPAR protein population and provide new insight into how spine structure can maintain synapse specificity by compartmentalizing lateral diffusion and therefore increasing the residence time of membrane proteins near individual synapses.