Traditional neuropsychological measurement of cognitive processing speed with tasks such as the Symbol Search and Coding subsets of the WAIS-IV, consistently show decline with advancing age. This is potentially problematic with populations where deficits in motor performance are expected, i.e., in aging or stroke populations. Thus, the aim of the current study was to explore the contribution of hand motor speed to traditional paper-and-pencil measures of processing speed and to a simple computer-customized non-motor perception decision task, the Inspection Time (IT) task. Participants were 67 young university students aged between 18 and 29 (59 females), and 40 older adults aged between 40 and 81 (31 females) primarily with a similar education profile. As expected, results indicated that age group differences were highly significant on the motor dexterity, Symbol Search and Coding tasks. However, no significant differences or correlations were seen between age groups and the simple visual perception IT task. Furthermore, controlling for motor dexterity did not remove significant age-group differences on the paper-and-pencil measures. This demonstrates that although much of past research into cognitive decline with age is confounded by use of motor reaction times as the operational measure, significant age differences in cognitive processing also exist on more complex tasks. The implications of the results are crucial in the realm of aging research, and caution against the use of traditional WAIS tasks with a clinical population where motor speed may be compromised, as in stroke.