The mammalian circadian (~24h) clock is based on a self-sustaining transcriptional-translational negative feedback loop (TTFL) centered around the PERIOD protein (PER), which is translated in the cytoplasm and then enters the nucleus to repress its own transcription at the right time of day. How such precise nucleus entry, critical for generating circadian rhythms, occurs is mysterious because thousands of PER molecules transit through crowded cytoplasm and arrive at the perinucleus across several hours. Here, we investigate this by developing a mathematical model that effectively describes the complex spatiotemporal dynamics of PER as a single random time delay. We find that the spatially coordinated bistable phosphoswitch of PER, which triggers the phosphorylation of accumulated PER at the perinucleus, can lead to the synchronous and precise nuclear entry of PER, and thus to precise transcriptional repression despite the heterogenous PER arrival times at the perinucleus. In particular, even when cell crowdedness, cell size, and transcriptional activator level change, and thus PER arrival times at the perinucleus are greatly perturbed, the bistable phosphoswitch allows the TTFL to maintain robust circadian rhythms. These results provide fundamental insight into how the circadian clock compensates for spatiotemporal noise from various intracellular sources.