Rydberg atoms, with their giant electronic orbitals, exhibit dipole-dipole interaction reaching the GHz range at a distance of a micron, making them a prominent contender for realizing quantum operations well within their coherence time. However, such strong interactions have never been harnessed so far, mainly because of the stringent requirements on the fluctuation of the atom positions and the necessary excitation strength. Here, using atoms trapped in the motional groundstate of optical tweezers and excited to a Rydberg state with picosecond pulsed lasers, we observe an interaction-driven energy exchange, i.e., a Förster oscilation, occuring in a timescale of nanoseconds, two orders of magnitude faster than in any previous work with Rydberg atoms. This ultrafast coherent dynamics gives rise to a conditional phase which is the key resource for an ultrafast controlled-Z gate. This opens the path for quantum simulation and computation operating at the speed-limit set by dipole-dipole interactions with this ultrafast Rydberg platform.