An Intelligent Robotic System Capable of Sensing and Describing Objects Based on Bimodal, Self‐Powered Flexible Sensors

Abstract

This study presents an intelligent soft robotic system capable of perceiving, describing, and sorting objects based on their physical properties. This work introduces a bimodal self‐powered flexible sensor (BSFS) based on the triboelectric nanogenerator and giant magnetoelastic effect. The BSFS features a simplified structure comprising a magnetoelastic conductive film and a packaged liquid metal coil. The BSFS can precisely detect and distinguish touchless and tactile models, with a response time of 10 ms. By seamlessly integrating the BSFSs into the soft fingers, this study realizes an anthropomorphic soft robotic hand with remarkable multimodal perception capabilities. The touchless signals provide valuable insights into object shape and material composition, while the tactile signals offer precise information regarding surface roughness. Utilizing a convolutional neural network (CNN), this study integrates all sensing information, resulting in an intelligent soft robotic system that accurately describes objects based on their physical properties, including materials, surface roughness, and shapes, with an accuracy rate of up to 97%. This study may lay a robotic foundation for the hardware of the general artificial intelligence with capacities to interpret and interact with the physical world, which also serves as an interface between artificial intelligence and soft robots.

Anisotropy in magnetic materials for sensors and actuators in soft robotic systems

“…17d and e). 279 In the touchless mode, as a negatively charged external object approaches the magnetoelastic conductive film, potential generation was induced from the charge generation based on contact electrification. Therefore, free electrons flow from the magnetoelastic conductive film to the ground, yielding current.…”

The role of bio-inspired micro-/nano-structures in flexible tactile sensors

“…6–10 The versatility and scalability of sensors in terms of materials and structures have led to significant growth in a variety of applications, including human motion monitoring, 11–14 disease diagnosis, 15–17 human–machine interactions, 18–21 and robot control. 22–24 Different from traditional metal or semiconductor sensors, which are limited by constraints such as rigidity and complex fabrication processes, flexible sensors are constructed from materials that can be deformed to fit a variety of irregular surfaces, making them suitable for a wider range of situations where external stimuli can be detected. 25–28 Among them, tactile sensors are expected to assist robots in realizing the perceptual capabilities of human skin, enabling them to interact with environment in a more natural and realistic way.…”

Self-Perceptional Soft Robotics by a Dielectric Elastomer

“…Meanwhile, pre-exposure perception, encompassing chemical, electromagnetic, auditory, and visual perception, holds paramount importance for organisms. Inspired by biology, a large number of precontact sensing techniques have been investigated in soft robots, such as ultrasonic sensing, 14,42 infrared sensing, 13 optical sensing, 43−45 biosensing, 46 laser sensing, 47 magnetic sensing, 48,49 and capacitive sensing. 10,50 Among these techniques, capacitive sensing in soft robotics has some significant advantages.…”