Spatial learning requires remembering and choosing paths to goals. Hippocampal place cells replay spatial paths during immobility in reverse and forward order, offering a potential mechanism. However, how replay mediates both goal-directed learning and memory-guided decision making is unclear. We therefore continuously tracked replay in the same hippocampalprefrontal ensembles throughout learning of a spatial alternation task. We found that during pauses between behavioral trajectories, awake reverse and forward hippocampal replay consistently mediated an internal cognitive search of all available past and future possibilities, and exhibited opposing learning gradients for prediction of past and future behavioral paths, respectively. Coordinated hippocampal-prefrontal replay mediated recall of correct past paths and selection of future choices leading to reward based on the hippocampal cognitive search, executing spatial working memory rules. Our findings reveal a learning shift from hippocampal reverse-replay-based retrospective evaluation to forward-replay-based prospective planning, with prefrontal filtering of memory-guided paths for learning and decision-making.