In this paper, we propose a multimodal flexible sensory interface for interactively teaching soft robots to perform skilled locomotion using bare human hands. First, we develop a flexible bimodal smart skin (FBSS) based on triboelectric nanogenerator and liquid metal sensing that can perform simultaneous tactile and touchless sensing and distinguish these two modes in real time. With the FBSS, soft robots can react on their own to tactile and touchless stimuli. We then propose a distance control method that enabled humans to teach soft robots movements via bare hand-eye coordination. The results showed that participants can effectively teach a self-reacting soft continuum manipulator complex motions in three-dimensional space through a “shifting sensors and teaching” method within just a few minutes. The soft manipulator can repeat the human-taught motions and replay them at different speeds. Finally, we demonstrate that humans can easily teach the soft manipulator to complete specific tasks such as completing a pen-and-paper maze, taking a throat swab, and crossing a barrier to grasp an object. We envision that this user-friendly, non-programmable teaching method based on flexible multimodal sensory interfaces could broadly expand the domains in which humans interact with and utilize soft robots.

Miniature soft sensors are crucial for the perception of soft robots. Although centimeter-scale sensors have been well developed, very few works addressed millimeter-scale, three-dimensional-shaped soft sensors capable of measuring multi-axis forces. In this work, we developed a millimeter-scale (overall size of 6 mm × 11 mm × 11 mm) soft sensor based on liquid metal printing technology and self-folding origami parallel mechanism. The origami design of the sensor enables the soft sensor to be manufactured within the plane and then fold into a three-dimensional shape. Furthermore, the parallel mechanism allows the sensor to rotate along two orthogonal axes. We showed that the soft sensor can be self-folded (took 17 s) using a shape-memory polymer and magnets. The results also showed that the sensor prototype can reach a deformation of up to 20 mm at the tip. The sensor can realize a measurement of external loads in six directions. We also showed that the soft sensor enables underwater sensing with a minimum sensitivity of 20 mm/s water flow. This work may provide a new manufacturing method and insight into future millimeter-scale soft sensors for bio-inspired robots.