According to the theory of embodied cognition, semantic processing is closely coupled with body movements. For example, constraining hand movements inhibits memory for objects that can be manipulated with the hands. However, it has not been confirmed whether body constraint reduces brain activity related to semantics. We measured the effect of hand constraint on semantic processing in the parietal lobe using functional near-infrared spectroscopy. A pair of words representing the names of hand-manipulable (e.g., cup or pencil) or nonmanipulable (e.g., windmill or fountain) objects were presented, and participants were asked to identify which object was larger. The reaction time (RT) in the judgment task and the activation of the left intraparietal sulcus (LIPS) and left inferior parietal lobule (LIPL), including the supramarginal gyrus and angular gyrus, were analyzed. We found that constraint of hand movement suppressed brain activity in the LIPS toward hand-manipulable objects and affected RT in the size judgment task. These results indicate that body constraint reduces the activity of brain regions involved in semantics. Hand constraint might inhibit motor simulation, which, in turn, would inhibit body-related semantic processing.