After decoupling, relic neutrinos traverse the evolving gravitational imhomogeneities along their trajectories. Once they turn non-relativistic, this results in a significant amplification of the anisotropies in the cosmic neutrino background (CνB). Past studies have reconstructed the phase-space distribution of relic neutrinos from the local distribution of matter (accounting for the Milky Way halo and the surrounding large-scale structures), but have neglected the CνB anisotropies in the initial conditions of neutrino trajectories. Using our previously developed N-1-body simulation framework, we show that including these primordial fluctuations in the initial conditions can be important, as it produces similar effects on the abundance and anisotropies of the CνB as the inclusion of large-scale structures beyond the Milky Way halo. Interpretability of data from future CνB observatories like PTOLEMY therefore depends on correctly modelling these effects. GitHub: our jax-accelerated simulation code can be found here.