Haptic Exploratory Behavior During Object Discrimination: A Novel Automatic Annotation Method

Jansen, Sander E. M.; Tiest, Wouter M. Bergmann; Kappers, Astrid M. L.

doi:10.1371/journal.pone.0117017

Cited by 12 publications

(10 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The possibility to combine this approach with quantitative information about the exploration directly provided by the device might help in designing automatic processes for annotation, thereby increasing reproducibility of the measures. This currently forms a crucial challenge in the context of haptic analysis [26], [27].…”

Section: Discussionmentioning

confidence: 99%

To Move or Not to Move: Development of Fine-Tuning of Object Motion in Haptic Exploration

Sciutti

Sandini

2022

IEEE Trans. Cogn. Dev. Syst.

View full text Add to dashboard Cite

Recognizing objects through touch is a complex task based on integrating information coming from multiple senses and motor commands guiding the exploratory motions. To gain insight into the development of exploratory strategies in children, in this study we addressed the question: how does exploration change when the stimulus becomes freely movable rather than fixed? We tested whether the possibility to move the object ushers in a strategic advantage, reducing the time and the number of touches necessary. We analyzed how school-aged children explore iCube, which is a sensorized cube measuring its orientation in space and contacts location. We tasked participants with finding specific cube faces; they could only touch the static cube, move and touch it, or move, touch and look at it. Visuo-haptic performances were adult-like at 7 years of age, whereas haptic exploration was less effective until 9 years. The fine-tuning of object movements as a function of task constraints, e.g. availability of vision or blind haptic task, increased significantly with age. Shedding light on how different factors shape haptic exploration could help researchers in the pursuit of detecting the occurrence of abnormal exploratory behaviors early on during development, providing a novel approach to detecting perceptual problems.

show abstract

Section: Discussionmentioning

confidence: 99%

To Move or Not to Move: Development of Fine-Tuning of Object Motion in Haptic Exploration

Sciutti

Sandini

2022

IEEE Trans. Cogn. Dev. Syst.

View full text Add to dashboard Cite

show abstract

“…Through this analysis, we could quantify the dynamic temporal features (e.g., frequency of touching-mode switching), which had never been found in conventional time-averaging analysis. Though we could estimate switching between touching modes by using the analysis, each touching mode itself could not be identified since methods for identifying touching modes are still in progress [29][30][31] . Developing an identification method for each switched touching mode in a trial is warranted to analyse participants' strategy in detail.…”

Section: Discussionmentioning

confidence: 99%

Frequency of Switching Touching Mode Reflects Tactile Preference Judgment

Yokosaka

Inubushi

Kuroki

et al. 2020

Sci Rep

View full text Add to dashboard Cite

with participants' hand motion regardless of the task, while the correlation for some other features (e.g., warmth) depended on it. It can be speculated that for some perceptual features, the bottom-up effect is dominant in hand motion, while for others, the top-down effect is. Since preference, the main topic of this study, may reflect combinations of multiple perceptual aspects, it is highly likely that hand motion to judge preference will change as a result of both effects. We hypothesized that studying the strategy used (i.e., how an object is explored) to judge preference for an object's surface will reveal how humans extract information about tactile preference. An earlier study, by showing that the averaged velocity of finger motion did not have a correlation with pleasantness ratings, suggested that there is no bottom-up effect on hand motion 4 . However, that study restricted participants' touching mode to stroking only; therefore, the effect of the explorative strategy, i.e., the top-down effect, on preference judgment remains unknown.Capturing the features of the touching mode from complex patterns of hand motion is challenging, since the touching mode category is not explicit in most cases and participants may use more than one mode. It is known that how participants touch an object is time-varying even within a single trial, where the tactile stimulus is not changed and physical properties are constant. For example, participants sequentially have used a couple of touching modes, such as grasping and stroking, to judge an object's properties [13][14][15][16] . This view is also supported by a constructive approach showing that a robot arm could identify a tactile stimulus with higher accuracy by sequentially using multiple touching modes 17,18 . Other studies have shown that the averaged velocity and force tends to increase within a single trial 8,19 . Clearly, we need a better index than a categorical name or temporally averaged value to capture the temporal dynamics of hand motion.Here, we introduce nonlinear time series analysis as an effective tool for understanding the temporal dynamics of explorative hand motion, and try to reveal the links between the observed temporal dynamics of unrestricted hand motion and preference rating. Nonlinear time series analysis is a well-established method often used for analyses of other kinds of explorative motion such as eye movement during scans of radiographs 20 and wielding-hand motion during judgment of rod length 21 . One of the useful tools in the nonlinear time series analysis is the recurrence plot (RP) technique, which provides insight into the dynamics graphically 22 . Moreover, by calculating some characteristic values from RPs, such as DET, Lmax, and TREND as described below, we can quantitatively study the randomness, predictability, and stationarity of the time series, which is referred to as recurrence quantification analysis. Our main finding is that the index of stationarity (TREND) had a positive correlation with preference rating. We also fou...

show abstract

“…We also posit that the use of simple noninvasive tools as the sensorized object described here could be in future used to augment the automatic assessment of haptic exploration strategies, reducing the need of manual annotation of videos, to increase reproducibility of the measures, a currently crucial challenge in the context of haptic analysis (Jansen et al, 2013, 2015). Indeed, the automatic extraction of the temporal dynamics of the exerted touches could allow reconstructing on average the effector’s motion features, together with the cube rotation.…”

Section: Discussionmentioning

confidence: 99%

“…As mentioned above, addressing this question is complex, as currently the characterization of the movements adopted during haptic exploration is performed mainly through lengthy manual annotations by human observers. This approach, beyond being fatiguing for the observers, might also be at risk of missing important exploratory behaviors due to drop in attention or to limitations in visibility (Jansen et al, 2015). It is, therefore, necessary to increase efficiency and reproducibility through new methods for automatic classifications (Jansen et al, 2013, 2015).…”

Section: Introductionmentioning

confidence: 99%

“…This approach, beyond being fatiguing for the observers, might also be at risk of missing important exploratory behaviors due to drop in attention or to limitations in visibility (Jansen et al, 2015). It is, therefore, necessary to increase efficiency and reproducibility through new methods for automatic classifications (Jansen et al, 2013, 2015). To this aim, we adopted a novel tool, iCube, a cube which measures its orientation in space and the location of the contacts on its faces and communicates this information wirelessly to a computer.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Visuo-Haptic Exploration for Multimodal Memory

Sciutti

Damonte

Alloisio

et al. 2019

Front. Integr. Neurosci.

View full text Add to dashboard Cite

When faced with a novel object, we explore it to understand its shape. This way we combine information coming from different senses, as touch, proprioception and vision, together with the motor information embedded in our motor execution plan. The exploration process provides a structure and constrains this rich flow of inputs, supporting the formation of a unified percept and the memorization of the object features. However, how the exploration strategies are planned is still an open question. In particular, is the exploration strategy used to memorize an object different from the exploration strategy adopted in a recall task? To address this question we used iCube, a sensorized cube which measures its orientation in space and the location of the contacts on its faces. Participants were required to explore the cube faces where little pins were positioned in varying number. Participants had to explore the cube twice and individuate potential differences between the two presentations, which could be performed either haptically alone, or with also vision available. The haptic and visuo-haptic (VH) exploratory strategies changed significantly when finalized to memorize the structure of the object with respect to when the same object was explored to recall and compare it with its memorized instance. These findings indicate that exploratory strategies are adapted not only to the property of the object to be analyzed but also to the prospective use of the resulting representation, be it memorization or recall. The results are discussed in light of the possibility of a systematic modeling of natural VH exploration strategies.

show abstract

Haptic Exploratory Behavior During Object Discrimination: A Novel Automatic Annotation Method

Cited by 12 publications

References 13 publications

To Move or Not to Move: Development of Fine-Tuning of Object Motion in Haptic Exploration

To Move or Not to Move: Development of Fine-Tuning of Object Motion in Haptic Exploration

Frequency of Switching Touching Mode Reflects Tactile Preference Judgment

Visuo-Haptic Exploration for Multimodal Memory

Contact Info

Product

Resources

About