Pushing the limits of remote RF sensing by reading lips under the face mask

Hameed, Hira; Usman, Muhammad; Tahir, Ahsen; Hussain, Amir; Abbas, Hasan; Cui, Tiejun; Imran, Muhammad Ali

doi:10.1038/s41467-022-32231-1

Cited by 26 publications

(19 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The work of UWB demonstrated the lip reading work with the vowels of A, E, I, O, U and static scenario, with a face mask. The result of 95% approves that the mouth motion brings informative signal for UWB sensing 7 . To expand the work and exploit more possibilities, we added words and sentences for data collection regarding the reference.…”

Section: Literature Survey Of Radar-enabled Speech Recognitionmentioning

confidence: 60%

A large-scale multimodal dataset of human speech recognition

Chong¹,

Li²,

Zhang³

et al. 2023

Preprint

View full text Add to dashboard Cite

Nowadays, non-privacy small-scale motion detection has attracted an increasing amount of research in remote sensing in speech recognition. These new modalities are employed to enhance and restore speech information from speakers of multiple types of data. In this paper, we propose a dataset contains 7.5 GHz Channel Impulse Response (CIR) data from ultra-wideband (UWB) radars, 77-GHz frequency modulated continuous wave (FMCW) data from millimetre wave (mmWave) radar, and laser data. Meanwhile, a depth camera is adopted to record the landmarks of the subject's lip and voice. Approximately 400 minutes of annotated speech profiles are provided, which are collected from 20 participants speaking 5 vowels, 15 words and 16 sentences. The dataset has been validated and has potential for the research of lip reading and multimodal speech recognition.

show abstract

Section: Literature Survey Of Radar-enabled Speech Recognitionmentioning

confidence: 60%

A large-scale multimodal dataset of human speech recognition

Chong¹,

Li²,

Zhang³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Gesture training is an important part of the GMI system, which is implemented by using VGG16 convolutional neural networks in this scheme, [45,46] and three synthetic image datasets of alphabet and some homemade data based on ASL are applied as the training samples. [47][48][49] The detailed gesture training framework based on VGG16 network can be seen in Section S1 (Supporting Information).…”

Section: Gesture Training Based On Vgg16 Networkmentioning

confidence: 99%

Computer‐Vision Based Gesture‐Metasurface Interaction System for Beam Manipulation and Wireless Communication

Chen,

Wang,

Sun

et al. 2023

Advanced Science

Self Cite

View full text Add to dashboard Cite

Hand gesture plays an important role in many circumstances, which is one of the most common interactive methods in daily life, especially for disabled people. Human–machine interaction is another popular research topic to realize direct and efficient control, making machines intelligent and maneuverable. Here, a special human–machine interaction system is proposed and namedas computer‐vision (CV) based gesture‐metasurface interaction (GMI) system, which can be used for both direct beam manipulations and real‐time wireless communications. The GMI system first needs to select its working mode according to the gesture command to determine whether to perform beam manipulations or wireless communications, and then validate the permission for further operation by recognizing unlocking gesture to ensure security. Both beam manipulation and wireless communication functions are validated experimentally, which show that the GMI system can not only realize real‐time switching and remote control of different beams through gesture command, but also communicate with a remote computer in real time by translating the gesture language to text message. The proposed non‐contact GMI system has the advantages of good interactivity, high flexibility, and multiple functions, which can find potential applications in community security, gesture‐command smart home, barrier‐free communications, and so on.

show abstract

“…Typically, 144 the turbo codes are often constructed with parallel concatenation of two recursive convolutional encoders that have been separated with interleaver 145 . Then, the task is designing the polynomials of the code for individual encoders, and for using a suitable interleaver.…”

Section: Channel Coding Over In‐vivomentioning

confidence: 99%

A survey on state of the inter body radio communication channel: Performance and solutions

Mezher¹,

Alabbas

Ilyas

2022

Trans Emerging Tel Tech

View full text Add to dashboard Cite

The development of in-vivo communications and networking systems has the potential for improving the healthcare delivery while also enabling the creation of new services and applications. In-vivo communications provide wireless networked cyber-physical systems of the embedded devices that enable accurate, quick, and cost-effective responses under a variety of circumstances. Also, it is focused upon modeling and characterizing in-vivo wireless channel, as well as contrasting it to other well-known channels. In this paper, channel coding is also mentioned because it improves data rates and performance gain. This work also discusses present problems and potential research topics for in-vivo communications. This article aims to show the researchers how the orthogonal frequency multiple access (OFDMA) technique improved the performance of in-vivo channel and how it is outperformed compared to the other techniques (TDM, TDMA, CDMA and OFDM) also how the OFDMA outperformed equalizers and channel coding in terms of bit error rate with giving a significant improvement of data transmitting and avoiding inter symbol interference. However, when it comes to the in-vivo communications, the modeling of in-vivo wireless channel is vital.Understanding the features of the in-vivo channel is critical for achieving optimal processing and designing successful procedures that allow WBANs to be arranged inside the human body.

show abstract

Pushing the limits of remote RF sensing by reading lips under the face mask

Cited by 26 publications

References 24 publications

A large-scale multimodal dataset of human speech recognition

A large-scale multimodal dataset of human speech recognition

Computer‐Vision Based Gesture‐Metasurface Interaction System for Beam Manipulation and Wireless Communication

A survey on state of the inter body radio communication channel: Performance and solutions

Contact Info

Product

Resources

About