Skin-inspired quadruple tactile sensors integrated on a robot hand enable object recognition

Abstract

Robot hands with tactile perception can improve the safety of object manipulation and also improve the accuracy of object identification. Here, we report the integration of quadruple tactile sensors onto a robot hand to enable precise object recognition through grasping. Our quadruple tactile sensor consists of a skin-inspired multilayer microstructure. It works as thermoreceptor with the ability to perceive thermal conductivity of a material, measure contact pressure, as well as sense object temperature and environment temperature simultaneously and independently. By combining tactile sensing information and machine learning, our smart hand has the capability to precisely recognize different shapes, sizes, and materials in a diverse set of objects. We further apply our smart hand to the task of garbage sorting and demonstrate a classification accuracy of 94% in recognizing seven types of garbage.

Micro‐Nano Processing of Active Layers in Flexible Tactile Sensors via Template Methods: A Review

“…Besides, benefiting from the general trend of multidisciplinary cross-integration, the intelligent robot grasp can not only grasp the object, but also achieve the accurate recognition of the grasped object. Zhu et al [197] proposed a quadruple tactile sensor integrated with pressure sensing, material thermal conductivity sensing, and bimodal temperature sensing. By using citric acid monohydrate particles as a template, a porous PDMS/Ag NPs material was prepared as a tactile sensing layer, and assembled with other components to form the tactile sensor, as shown in Figure 13c.…”

Artificial Intelligence of Things (AIoT) Enabled Virtual Shop Applications Using Self‐Powered Sensor Enhanced Soft Robotic Manipulator

“…Li et al have reported a quadruple tactile sensor capable of pressure sensing, material thermal conductivity sensing, and bimodal temperature sensing, so that the shape, material, and texture of objects can be perceived through the fusion of multiple sensing information. [72] Wan et al have proposed a bimodal artificial sensory neuron based on ionic/electronic hybrid neuromorphic electronics to implement the visual-haptic fusion, providing multidimensional spatial information for the robotic hand. [73] However, many of these mentioned methods are still confronted with challenges in cost effectiveness, energy consumption, compatibility of materials and process.…”

“…[ 112 ] However, for realizing more complex functions, e.g., objects recognition, to enhance the intelligence of anthropomorphic robots in virtual shop application, the multimodal sensing data generated from different sensors are needed to be further interpreted and integrated with ML technology, especially for the concerns of identifying different objects with similar shapes, or the same objects with different grasping direction. [ 72 , 113 ] In addition, the pyroelectric‐based temperature perception is also important for soft robots to enhance the intelligence of object recognition, such as identifying the cold or hot drink. Overall, as an instance of the virtual shop, an advanced interactive system with both the intelligent soft manipulator with the multifunctional sensory system, and the immersive virtual interface which offers comprehensive information of the products in real space, will greatly improve users' experience.…”

Fruit Classification Utilizing a Robotic Gripper with Integrated Sensors and Adaptive Grasping

“…However, because the lighting conditions are frequently unsatisfactory in agricultural applications, vision systems cannot accurately recognize the objects even when sophisticated vision algorithms are used in certain cropping environments [1,24]. Integrated tactile sensors are key components of a robot hand system, which provide grippers with the ability to interact with their surroundings precisely, rapidly, and safely [25,26]. e use of tactile sensing perception in robots can improve the safety of object manipulation and the efficiency of object identification.…”