The arrival directions of TeV-PeV cosmic rays are remarkably uniform due to the isotropization of their directions by scattering on turbulent magnetic fields. Small anisotropies can exist in standard diffusion models, however, only on the largest angular scales. Yet, high-statistics observatories like IceCube and HAWC have found significant deviations from isotropy down to small angular scales. Here, we explain the formation of small-scale anisotropies by considering pairs of cosmic rays that get correlated by their transport through the same realisation of the turbulent magnetic field. We argue that the formation of small-scale anisotropies is the reflection of the particular realisation of the turbulent magnetic field experienced by cosmic rays on time scales intermediate between the early, ballistic regime and the late, diffusive regime. We approach this problem in two different ways: First, we run test particle simulations in synthetic turbulence, covering for the first time the TV rigidities of observations with realistic turbulence parameters. Second, we extend the recently introduced mixing matrix approach and determine the steady-state angular power spectrum. Throughout, we adopt magneto-static, slab-like turbulence. We find excellent agreement between the predicted angular power spectra in both approaches over a large range of rigidities. In the future, measurements of small-scale anisotropies will be valuable in constraining the nature of the turbulent magnetic field in our Galactic neighborhood.