The human ability to perform joint actions is often attributed to high-level cognitive processes. For example, the finding that action leaders act faster when imitated by their partners has been interpreted as evidence for anticipation of the other’s actions (Pfister, Dignath, Hommel, & Kunde, 2013). In two experiments, we showed that a low-level mechanism can account for this finding. Action leaders were faster when imitated than when counterimitated, but only if they could observe their partner’s actions (Exp. 1). Crucially, when due to our manipulation the partner’s imitative actions became slower than the counterimitative actions, leaders also became slower when they were imitated, and faster when counterimitated (Exp. 2). Our results suggest that spontaneous temporal adaptation is a key mechanism in joint action tasks. We argue for a reconsideration of other phenomena that have traditionally been attributed solely to high-level processes.Electronic supplementary materialThe online version of this article (doi:10.3758/s13423-016-1141-3) contains supplementary material, which is available to authorized users.