We present a theory for the memory effect in electron glasses. In fast gate voltage sweeps it is manifested as a dip in the conductivity around the equilibration gate voltage. We show that this feature, also known as anomalous field effect, arises from the long-time persistence of correlations in the electronic configuration. We argue that the gate voltage at which the memory dip saturates is related to an instability caused by the injection of a critical number of excess carriers. This saturation threshold naturally increases with temperature. On the other hand, we argue that the gate voltage beyond which memory is erased, is temperature independent. Using standard percolation arguments, we calculate the anomalous field effect as a function of gate voltage, temperature, carrier density and disorder. Our results are consistent with experiments, and in particular, they reproduce the observed scaling of the width of the memory dip with various parameters.