The navigational skills of ants, bees and wasps represent one of the most baffling examples of the powers of minuscule brains. Insects store long-term memories of the visual scenes they experience, and they use compass cues to build a robust representation of directions. We know reasonably well how long-term memories are formed, in a brain area called the Mushroom Bodies (MB), as well as how heading representations are formed in another brain area called the Central Complex (CX). However, how such memories and heading representations interact to produce powerful navigational behaviours remains unclear. Here we combine behavioural experiments with computational modelling that is strictly based on connectomic data to provide a new perspective on how navigation might be orchestrated in these insects. Our results reveal a lateralised design, where signals about whether to turn left or right are segregated in the left and right hemispheres, respectively. Furthermore, we show that guidance is a two-stage process: the recognition of visual memories, presumably in the MBs, does not directly drive the motor command, but instead updates a desired heading, presumably in the CX, which in turn is used to control guidance using celestial compass information. Overall, this circuit enables ants to recognise views independently of their body orientation, and combines terrestrial and celestial cues in a way that produces exceptionally robust navigation.