The efficient conversion of spin to charge transport and vice versa is of major relevance for the detection and generation of spin currents in spin‐based electronics. Interfaces of heterostructures are known to have a marked impact on this process. Here, terahertz (THz) emission spectroscopy is used to study ultrafast spin‐to‐charge‐current conversion (S2C) in about 50 prototypical F|N bilayers consisting of a ferromagnetic layer F (e.g., Ni81Fe19, Co, or Fe) and a nonmagnetic layer N with strong (Pt) or weak (Cu and Al) spin‐orbit coupling. Varying the structure of the F/N interface leads to a drastic change in the amplitude and even inversion of the polarity of the THz charge current. Remarkably, when N is a material with small spin Hall angle, a dominant interface contribution to the ultrafast charge current is found. Its magnitude amounts to as much as about 20% of that found in the F|Pt reference sample. Symmetry arguments and first‐principles calculations strongly suggest that the interfacial S2C arises from skew scattering of spin‐polarized electrons at interface imperfections. The results highlight the potential of skew scattering for interfacial S2C and propose a promising route to enhanced S2C by tailored interfaces at all frequencies from DC to terahertz.