Looking to recognise: the pre-eminence of semantic over sensorimotor processing in human tool use

Abstract

Alongside language and bipedal locomotion, tool use is a characterizing activity of human beings. Current theories in the field embrace two contrasting approaches: "manipulation-based" theories, which are anchored in the embodied-cognition view, explain tool use as deriving from past sensorimotor experiences, whereas "reasoning-based" theories suggest that people reason about object properties to solve everyday-life problems. Here, we present results from two eye-tracking experiments in which we manipulated the visuo-perceptual context (thematically consistent vs. inconsistent object-tool pairs) and the goal of the task (free observation or looking to recognise). We found that participants exhibited reversed tools' visual-exploration patterns, focusing on the tool's manipulation area under thematically consistent conditions and on its functional area under thematically inconsistent conditions. Crucially, looking at the tools with the aim of recognising them produced longer fixations on the tools' functional areas irrespective of thematic consistency. In addition, tools (but not objects) were recognised faster in the thematically consistent conditions. These results strongly support reasoning-based theories of tool use, as they indicate that people primarily process semantic rather than sensorimotor information to interact with the environment in an agent's consistent-with-goal way. Such a pre-eminence of semantic processing challenges the mainstream embodied-cognition view of human tool use.

Effects of Tool Novelty and Action Demands on Gaze Searching During Tool Observation

“…Interestingly, these results above seem to contradict prior studies; two research found that participants gazed more at the grasping part when they could freely observe a familiar tool and an object with a consistent combination ( Federico and Brandimonte, 2019 , 2020 ). However, in one of these studies, when the combination was consistent and presented at a spatially distant location, participants gazed more at the functional part of the tool ( Federico and Brandimonte, 2019 ), namely, if the tool and the combined object are not in a spatial position where they can be manipulated, people may not automatically imagine the action of using the tool.…”

“…Furthermore, in our study, we did not present another object in combination with the tool; instead, we presented only the tool. This experimental condition corresponds methodologically to the free viewing condition of Federico and Brandimonte (2020) , in which the object and the tool are inconsistently combined. Moreover, the short-term recognition task of tools (or objects) present in Federico and Brandimonte (2020)’ s study corresponds methodologically to the use condition in the current study, since it involves the process of recalling how to use the tools (or objects).…”

“…This experimental condition corresponds methodologically to the free viewing condition of Federico and Brandimonte (2020) , in which the object and the tool are inconsistently combined. Moreover, the short-term recognition task of tools (or objects) present in Federico and Brandimonte (2020)’ s study corresponds methodologically to the use condition in the current study, since it involves the process of recalling how to use the tools (or objects). The results of these conditions in Federico and Brandimonte (2020)’ s study are consistent with the results of the free viewing and use conditions in our study – an increase in gaze on the functional parts of the tools.…”

“…Moreover, the short-term recognition task of tools (or objects) present in Federico and Brandimonte (2020)’ s study corresponds methodologically to the use condition in the current study, since it involves the process of recalling how to use the tools (or objects). The results of these conditions in Federico and Brandimonte (2020)’ s study are consistent with the results of the free viewing and use conditions in our study – an increase in gaze on the functional parts of the tools. Therefore, our results are consistent with those of Federico and Brandimonte (2020)’ s study.…”

“…A study on gaze response showed that, in familiar daily tools with a functional and grasping part separated along the long axis, the first fixation was biased toward the center and the next was biased toward the functional part; this fixation bias on the functional part was deemed a higher-order affordance effect, based on functional knowledge of tools ( Van Der Linden et al, 2015 ). Other reports on the free observation of the combination of familiar tools and objects showed that, when the combination was consistent, participants gazed more the grasping part of the tool, whereas when the combination was inconsistent, participants gazed more its functional part ( Federico and Brandimonte, 2019 , 2020 ). However, the characteristics of spatial attention or gaze response when observing an unfamiliar novel tool without the intention to use it remain unclear.…”