Observing other people acting activates imitative motor plans in the observer. Whether, and if so when and how, such ‘effector-specific motor simulation’ contributes to action recognition remains unclear. We report that individuals born without upper limbs (IDs)—who cannot covertly imitate upper-limb movements—are significantly less accurate at recognizing degraded (but not intact) upper-limb than lower-limb actions (i.e. point-light animations). This finding emphasizes the need to reframe the current controversy regarding the role of effector-specific motor simulation in action recognition: instead of focusing on the dichotomy between motor and non-motor theories, the field would benefit from new hypotheses specifying when and how effector-specific motor simulation may supplement core action recognition processes to accommodate the full variety of action stimuli that humans can recognize.