Soft Robotic Link with Controllable Transparency for Vision-based Tactile and Proximity Sensing

Abstract

Robots have been brought to work close to humans in many scenarios. For coexistence and collaboration, robots should be safe and pleasant for humans to interact with. To this end, the robots could be both physically soft with multimodal sensing/perception, so that the robots could have better awareness of the surrounding environment, as well as to respond properly to humans' action/intention. This paper introduces a novel soft robotic link, named ProTac, that possesses multiple sensing modes: tactile and proximity sensing, based on computer vision and a functional material. These modalities come from a layered structure of a soft transparent silicon skin, a polymer dispersed liquid crystal (PDLC) film, and reflective markers. Here, the PDLC film can switch actively between the opaque and the transparent state, from which the tactile sensing and proximity sensing can be obtained by using cameras solely built inside the ProTac link. In this paper, inference algorithms for tactile proximity perception are introduced. Evaluation results of two sensing modalities demonstrated that, with a simple activation strategy, ProTac link could effectively perceive useful information from both approaching and in-contact obstacles. The proposed sensing device is expected to bring in ultimate solutions for design of robots with softness, whole-body and multimodal sensing, and safety control strategies.

Marker Displacement Method Used in Vision-Based Tactile Sensors—From 2-D to 3-D: A Review

“…Muscularis [93], TacLINK [94], and IoTouch [95] used two cameras, which were arranged at the top and bottom of the robot link, to capture the 3D displacement of the marker array on the inner wall. Such designs were also used in the ProTac sensor with proximity perception function [96]. The Tac3D sensor [23], [97] realized simultaneous measurement of two virtual cameras from different angles through a monocular camera, which was achieved by arranging mirrors to construct two optical paths.…”

“…This technology could be extended to dexterous manipulation, human-computer interaction and other fields. The ProTac robotic link combined 3D MDM based on DNN model and proximity sensing technology, and the proposed design could be extended to other types of tactile sensing elements, and further extended to robot arm applications [96]. GelStereo uses point-cloud registration [25] and neural network [92], [105] to handle different application scenarios, including in-hand object localization and insertion, and adaptive capture.…”