Long term synaptic plasticity, such as long term potentiation (LTP), has been widely accepted as a cellular mechanism underlying memory. Recently, it has been unraveled that Shp2 plays a role in synaptic plasticity and memory in Drosophila and mice, revealing significant and conserved effects of Shp2 in cognitive function. However, the exact mechanism underlying this function of Shp2 in synaptic plasticity and memory still remains elusive. Here, we examine the regulation of Shp2 in hippocampal LTP and contextual fear conditioning. We find that Shp2 is rapidly recruited into spines after LTP induction. Furthermore, the phosphorylation level of Shp2 at Tyr-542 is elevated after LTP stimuli either in cultured hippocampal neurons or acute slices. Notably, contextual fear conditioning also regulates the phosphorylation level of Shp2 at Tyr-542, suggesting fine-tuned regulation of Shp2 in LTP and memory formation. By using a Shp2-specific inhibitor and adeno-associated virus-Cre mediated Shp2 knock-out in cultured neurons, we provide evidence that the phosphatase activity of Shp2 is critical for activity-dependent AMPA receptor surface trafficking. Collectively, our results have revealed a regulatory mechanism of Shp2 underlying LTP and memory, broadening our understanding of Shp2 in cognitive function.