In the marginal ice zone, surface waves drive motion of sea ice floes. The motion of floes relative to each other can cause periodic collisions, and drives the formation of pancake sea ice. Additionally, the motion of floes relative to the water results in turbulence generation at the interface between the ice and ocean below. These are important processes for the formation and growth of pancakes, and likely contribute to wave energy loss. Models and laboratory studies have been used to describe these motions, but there have been no in situ observations of relative ice velocities in a natural wave field. Here, we use shipboard stereo video to measure wave motion and relative motion of pancake floes simultaneously. The relative velocities of pancake floes are typically small compared to wave orbital motion (i.e. floes mostly follow the wave orbits). We find that relative velocities are well-captured by existing phase-resolved models, and are only somewhat over-estimated by using bulk wave parameters. Under the conditions observed, estimates of wave energy loss from ice–ocean turbulence are much larger than from pancake collisions. Increased relative pancake floe velocities in steeper wave fields may then result in more wave attenuation by increasing ice–ocean shear.