Functional mimicry of Ruffini receptors with fibre Bragg gratings and deep neural networks enables a bio-inspired large-area tactile-sensitive skin

Abstract

Collaborative robots are expected to physically interact with humans in daily living and the workplace, including industrial and healthcare settings. A key related enabling technology is tactile sensing, which currently requires addressing the outstanding scientific challenge to simultaneously detect contact location and intensity by means of soft conformable artificial skins adapting over large areas to the complex curved geometries of robot embodiments. In this work, the development of a large-area sensitive soft skin with a curved geometry is presented, allowing for robot total-body coverage through modular patches. The biomimetic skin consists of a soft polymeric matrix, resembling a human forearm, embedded with photonic fibre Bragg grating transducers, which partially mimics Ruffini mechanoreceptor functionality with diffuse, overlapping receptive fields. A convolutional neural network deep learning algorithm and a multigrid neuron integration process were implemented to decode the fibre Bragg grating sensor outputs for inference of contact force magnitude and localization through the skin surface. Results of 35 mN (interquartile range 56 mN) and 3.2 mm (interquartile range 2.3 mm) median errors were achieved for force and localization predictions, respectively. Demonstrations with an anthropomorphic arm pave the way towards artificial intelligence based integrated skins enabling safe human–robot cooperation via machine intelligence.

Localizing strain via micro-cage structure for stretchable pressure sensor arrays with ultralow spatial crosstalk

“…Development of large-scale and high-density flexible sensor arrays could provide better human–machine interaction 1 – 8 , and high-precision tactile perception could be achieved by analyzing the measured data 9 , 10 or isolating different crosstalk 11 – 16 . At present, signal crosstalk in sensor arrays is mainly divided into two types.…”

“…An ultra-light and imperceptible plastic electronics 53 has been demonstrated for pressure imaging and could be applied on more complex surfaces such as oral cavity, these higher requirements also aggravate the mechanical crosstalk in flexible electronics. Furthermore, Oddo 10 et al reported a biomimetic skin based on the photonic fiber Bragg grating transducers with overlapping receptive fields, and the force and localization predictions could be realized by the convolution neural deep learning algorithm. This technology combined with an efficient calculation power system will show broad application prospects in the field of collaborative robots.…”

2D pressure sensing map based on distributed sensing approach for biomedical application

“…With the help of distributed fiber sensing, it is possible to construct highly resolved distributed measurements of temperature 8,9 , strain, pressure, and refractive index 10 . Regarding strain and force mapping, applications including skin care during prosthetic treatment 11 and tactile sensing in human-robot interfaces have benefited greatly from fiber optic sensing systems 12 . It is of significant importance to perform human bite force measurement using distributed FOS to measure at the highest spatial resolution 13 .…”

Technology Roadmap for Flexible Sensors

“…37−39 For robots to interact safely with humans, 40 high densities of sensors (>10 sensors per cm 2 to be comparable to human skin 41 ) would need to cover the curved robotic surface over large areas (∼m 2 ). 42 In this regard, monolithically manufacturing sensor arrays on flexible substrates is much more efficient than individually placing rigid sensor pixels. 43,44 Furthermore, smart packaging with embedded sensors for product tracking and quality monitoring will be critical to efficient and sustainable supply chains, 45 yet sensor stickers need to be manufactured at costs as low as a few cents to realize this large-scale (and often disposable) application.…”

“…Medical-grade measurements of electrocardiogram and sweat metabolites require conformal contact between the sensor and the skin, but this goal is hardly achievable by rigid sensors without causing discomfort due to the surface micro-texture and deformation experienced by the skin . In contrast, soft and stretchable sensors can address these issues, offering disruptive solutions for future healthcare. − For robots to interact safely with humans, high densities of sensors (>10 sensors per cm 2 to be comparable to human skin) would need to cover the curved robotic surface over large areas (∼m 2 ) . In this regard, monolithically manufacturing sensor arrays on flexible substrates is much more efficient than individually placing rigid sensor pixels. , Furthermore, smart packaging with embedded sensors for product tracking and quality monitoring will be critical to efficient and sustainable supply chains, yet sensor stickers need to be manufactured at costs as low as a few cents to realize this large-scale (and often disposable) application.…”