Recognizing social touch gestures using recurrent and convolutional neural networks

Hughes, Dana; Krauthammer, Alon; Correli, Nikolaus

doi:10.1109/icra.2017.7989267

Cited by 26 publications

(16 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zheng et al [81] further studied full convolutional network learning. A hybrid deep learning structure with a CNN and a recurrent neural network were designed to process sequential tactile information online and solve the tactile emotion recognition problem in the human-robot interaction process [82]. Sohn et al [83] applied deep learning to large-scale electronic skin tactile perception.…”

Section: Tactile Feature Learning and Classificationmentioning

confidence: 99%

Embodied tactile perception and learning

Liu

Guo

Sun

et al. 2020

Brain Science Advances

View full text Add to dashboard Cite

Various living creatures exhibit embodiment intelligence, which is reflected by a collaborative interaction of the brain, body, and environment. The actual behavior of embodiment intelligence is generated by a continuous and dynamic interaction between a subject and the environment through information perception and physical manipulation. The physical interaction between a robot and the environment is the basis for realizing embodied perception and learning. Tactile information plays a critical role in this physical interaction process. It can be used to ensure safety, stability, and compliance, and can provide unique information that is difficult to capture using other perception modalities. However, due to the limitations of existing sensors and perception and learning methods, the development of robotic tactile research lags significantly behind other sensing modalities, such as vision and hearing, thereby seriously restricting the development of robotic embodiment intelligence. This paper presents the current challenges related to robotic tactile embodiment intelligence and reviews the theory and methods of robotic embodied tactile intelligence. Tactile perception and learning methods for embodiment intelligence can be designed based on the development of new large‐scale tactile array sensing devices, with the aim to make breakthroughs in the neuromorphic computing technology of tactile intelligence.

show abstract

Section: Tactile Feature Learning and Classificationmentioning

confidence: 99%

Embodied tactile perception and learning

Liu

Guo

Sun

et al. 2020

Brain Science Advances

View full text Add to dashboard Cite

show abstract

“…Utilizing a common set of low-dimensional features for each of the desired tasks, as opposed to using possibly distinct features specific to each task, reduces memory and computing requirements, allowing for material-scale computing elements to be used. This becomes especially relevant when requiring more complex models to generate features, such as the CNN used by Hughes et al (2017) or calculating spectral components (Hughes and Correll, 2015).…”

Section: Smart Skinmentioning

confidence: 99%

“…This section describes two example robotic materials in more detail: a smart tire capable of identifying terrain for use with autonomous vehicles (Dana Hughes and Correll, 2017), and a robotic skin capable of jointly performing gesture recognition and obstacle avoidance . These examples are used to demonstrate common attributes of robotic materials: processing high-bandwidth sensor information, using material-scale computing elements with limited resources, and the desire for efficient approaches to generating low-dimensional internal states or responses.…”

Section: Example Applicationsmentioning

confidence: 99%

Materials that make robots smart

Hughes

Heckman

Correll

2019

The International Journal of Robotics Research

Self Cite

View full text Add to dashboard Cite

We posit that embodied artificial intelligence is not only a computational, but also a materials problem. While the importance of material and structural properties in the control loop are well understood, materials can take an active role during control by tight integration of sensors, actuators, computation, and communication. We envision such materials to abstract functionality, therefore making the construction of intelligent robots more straightforward and robust. For example, robots could be made of bones that measure load, muscles that move, skin that provides the robot with information about the kind and location of tactile sensations ranging from pressure to texture and damage, eyes that extract high-level information, and brain material that provides computation in a scalable manner. Such materials will not resemble any existing engineered materials, but rather the heterogeneous components out of which their natural counterparts are made. We describe the state-of-the-art in so-called “robotic materials,” their opportunities for revolutionizing applications ranging from manipulation to autonomous driving by describing two recent robotic materials, a smart skin and a smart tire in more depth, and conclude with open challenges that the robotics community needs to address in collaboration with allies, such as wireless sensor network researchers and polymer scientists.

show abstract

“…Deep learning approaches require no feature engineering step and give similar results compared to feature-engineering based systems [18]. The state of art results were obtained using 2D-CNN for CoST [2] (author did not follow the challenge protocol) and a 3D-CNN for HAART [30].…”

Section: Introductionmentioning

confidence: 99%

Touch Recognition with Attentive End-to-End Model

Bani

Chétouani

2020

Proceedings of the 2020 International Conference on Multimodal Interaction

View full text Add to dashboard Cite

Touch is the earliest sense to develop and the first mean of contact with the external world. Touch also plays a key role in our socioemotional communication: we use it to communicate our feelings, elicit strong emotions in others and modulates behavior (e.g. compliance). Although its relevance, touch is an understudied modality in Human-Machine Interaction compared to audition and vision. Most of the social touch recognition systems require a feature engineering step making them difficult to compare and to generalize to other databases. In this paper, we propose an end-to-end approach. We present an attention-based end-to-end model for touch gesture recognition evaluated on two public datasets (CoST and HAART) in the context of the ICMI 15 Social Touch Challenge. Our model gave a similar level of accuracy: 61% for CoST and 68% for HAART and uses self-attention as an alternative to feature engineering and Recurrent Neural Networks. CCS CONCEPTS • Human-centered computing → Human computer interaction (HCI); • Computing methodologies → Machine learning.

show abstract

Recognizing social touch gestures using recurrent and convolutional neural networks

Cited by 26 publications

References 15 publications

Embodied tactile perception and learning

Embodied tactile perception and learning

Materials that make robots smart

Touch Recognition with Attentive End-to-End Model

Contact Info

Product

Resources

About