Learned Integrated Sensing Pipeline: Reconfigurable Metasurface Transceivers as Trainable Physical Layer in an Artificial Neural Network

Abstract

The rapid proliferation of intelligent systems (e.g., fully autonomous vehicles) in today's society relies on sensors with low latency and computational effort. Yet current sensing systems ignore most available a priori knowledge, notably in the design of the hardware level, such that they fail to extract as much task‐relevant information per measurement as possible. Here, a “learned integrated sensing pipeline” (LISP), including in an end‐to‐end fashion both physical and processing layers, is shown to enable joint learning of optimal measurement strategies and a matching processing algorithm, making use of a priori knowledge on task, scene, and measurement constraints. Numerical results demonstrate accuracy improvements around 15% for object recognition tasks with limited numbers of measurements, using dynamic metasurface apertures capable of transceiving programmable microwave patterns. Moreover, it is concluded that the optimal learned microwave patterns are nonintuitive, underlining the importance of the LISP paradigm in current sensorization trends.

Intelligent Electromagnetic Sensing with Learnable Data Acquisition and Processing

“…Recently, inspired by pioneering work in optical microscopy, 21 the idea of learned EM sensing with programmable metasurface hardware was introduced. 22 Thereby, a model of the programmable measurement process is directly integrated into the ML pipeline used to process the data, enabling the joint learning of optimal measurement and processing settings for the given hardware, task, and expected scene. We note that there is a number of related works in optics 23 , 24 , 25 , 26 , 27 and a similar concept has recently also been studied in the context of ultrasonic imaging.…”

“…Hence, for tasks such as object recognition it is most efficient to skip the intermediate imaging step and directly process the raw data, as done in previous studies. 20 , 22 …”

Metasurface-assisted massive backscatter wireless communication with commodity Wi-Fi signals

“…The power needed to program the metasurface is minimal and can be as low as a few μW per meta-atom 24 . By now, programmable metasurfaces have found various valuable applications, for instance in programmable electromagnetic imaging and sensing [25][26][27][28][29][30][31] , wireless communication 8,[32][33][34][35][36][37] , dynamic holograms 38 , wireless energy deposition 39,40 , and analog computation with indoor Wi-Fi infrastructure 41 . The proposed MBWC paradigm utilizes the programmable metasurface for three major purposes: (1) encoding the digital information to be conveyed on the physical level; (2) directly modulating the ambient stray electromagnetic waves with high signal-to-noise ratio (SNR); and (3) facilitating the retrieval of digital information encoded into the metasurface with a matching classifier or decoder.…”

A Comprehensive Review of Micro-Inertial Measurement Unit Based Intelligent PIG Multi-Sensor Fusion Technologies for Small-Diameter Pipeline Surveying

“…In addition, with the rapid development and application of the Artificial Intelligent (AI) technology, the AI-related intelligent technology is also a potential research direction for the optimal estimation of the multi-sensor-fused intelligent PIG surveying system [ 96 , 97 , 98 , 99 ].…”