Several studies indicate the functional importance of the motor cortex for higher cognition, language and semantic processing, and place the neural substrate of these processes in sensorimotor action-perception circuits linking motor, sensory and perisylvian language regions. Interestingly, in individuals with autism spectrum disorder (ASD), semantic processing of action and emotion words seems to be impaired and is associated with hypoactivity of the motor cortex during semantic processing. In this study, the relationship between semantic processing, fine motor skills and clinical symptoms was investigated in 19 individuals with ASD and 22 typically-developing matched controls. Participants completed two semantic decision tasks involving words from different semantic categories, a test of alexithymia (the Toronto Alexithymia Scale), and a test of fine motor skills (the Purdue Pegboard Test). A significant Group × Word Category interaction in accuracy ( p < 0.05) demonstrated impaired semantic processing for action words, but not object words in the autistic group. There was no significant group difference when processing abstract emotional words or abstract neutral words. Moreover, our study revealed deficits in fine motor skills as well as evidence for alexithymia in the ASD group, but not in neurotypical controls. However, these motor deficits did not correlate significantly with impairments in action-semantic processing. We interpret the data in terms of an underlying dysfunction of the action-perception system in ASD and its specific impact on semantic language processing.