Attosecond photoionization time delays reveal information about the potential energy landscape that an outgoing electron wavepacket probes upon ionization. In this study, we experimentally quantify the dependence of the time delay on the angular momentum of the liberated photoelectrons. For this purpose, we resolved electron quantum-path interference spectra in energy and angle using a two-color attosecond pump-probe photoionization experiment in helium. A fitting procedure of the angle-dependent interference pattern allows us to disentangle the relative phase of all four quantum pathways that are known to contribute to the final photoelectron signal. In particular, we resolve the dependence on angular momentum of the delay of one-photon transitions between continuum states, which is an essential and universal contribution to the total photoionization delay observed in attosecond pump-probe measurements. For such continuum-continuum transitions, we measure a delay between outgoing s and d electrons as large as 12 attoseconds, close to the ionization threshold in helium. Both single-active-electron and first-principles ab initio simulations confirm this observation for helium and hydrogen, demonstrating the universality of the observed delays. Published by The Optical Society under the terms of the Creative Commons Attribution 4.0 License. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.