Recent work has shown the fundamental role that cognitive strategies play in visuomotor adaptation. While algorithmic strategies, such as mental rotation, are flexible and generalizable, they are computationally demanding. To avoid this computational cost, people can instead rely on memory retrieval of a previously successful strategy. However, they are likely subject to strict stimulus-response associations and rely heavily on working memory capacity. In the current work, we sought to reveal the constraints of retrieval strategies in visuomotor adaptation. In a series of five experiments, we sought to estimate the working memory capacity limitations of retrieval strategies for visuomotor adaptation. This was accomplished by leveraging different variations of visuomotor item-recognition and visuomotor rotation recall tasks where we associated unique rotations with specific targets in the workspace and manipulated the set size (i.e., number of rotation-target associations). Notably, from Experiment 1 to 4, we found key signatures of working memory retrieval and not mental rotation. In particular, participants were less accurate and slower for larger set sizes. Importantly, we estimated that participants’ capacity in our experiments was between 2-5 targets across experiments. In addition, we found clear recency effects where reaching movements were more accurate and faster for more recent items. These results were corroborated using model-based metrics from a Bayesian latent-mixture analysis over target error distributions. Finally, in Experiment 5, we showed how the constraints observed across Experiments 1 to 4 can be overcome when relying on long-term memory retrieval. Our results point to the opportunity of studying other sources of memories where visuomotor solutions can be stored (e.g., episodic memories) to achieve successful adaptation.