A Robust Sign Language Recognition System with Sparsely Labeled Instances Using Wi-Fi Signals

Shang, Jiacheng; Wu, Jie

doi:10.1109/mass.2017.41

Cited by 21 publications

(14 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…as a weighting, the selection is made based on a defined threshold. Shang and Wu [156] use a selection method based on feature discretization. After discretizing all feature values, they consider instances as sufficiently similar if they share the same label and the same discretized feature value for each dimension.…”

Section: F Instance-based Transfermentioning

confidence: 99%

Transfer Learning With Time Series Data: A Systematic Mapping Study

Weber

Auch

Doblander³

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Transfer Learning is a well-studied concept in machine learning, that relaxes the assumption that training and testing data need to be drawn from the same distribution. Recent success in applying transfer learning in the area of computer vision has motivated research on transfer learning also in context of time series data. This benefits learning in various time series domains, including a variety of domains based on sensor values. In this paper, we conduct a systematic mapping study of literature on transfer learning with time series data. Following the review guidelines of Kitchenham and Charters, we identify and analyze 223 relevant publications. We describe the pursued approaches and point out trends. Especially during the last two years, there has been a vast increase in the number of publications on the topic. This paper's findings can help researchers as well as practitioners getting into the field and can help identify research gaps.

show abstract

Section: F Instance-based Transfermentioning

confidence: 99%

Transfer Learning With Time Series Data: A Systematic Mapping Study

Weber

Auch

Doblander³

et al. 2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…For example, MyoSign uses EMG sensors [57] and DeepASL [14] requires a Leap Motion [50] depth sensor. Second, the use of infrastructure-embedded sensors (e.g., WiFi [34,45,56], mmWave radios [44]) require the users to perform hand gestures in pre-defined locations. In addition, while sign language consist of more than the hand gestures themselves [15], most previous works have only focused on the gestural characteristics (i.e., movements) of the hands.…”

Section: :2 • Park Et Almentioning

confidence: 99%

Enabling Real-time Sign Language Translation on Mobile Platforms with On-board Depth Cameras

Park

Lee

2021

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.

View full text Add to dashboard Cite

In this work we present SUGO, a depth video-based system for translating sign language to text using a smartphone's front camera. While exploiting depth-only videos offer benefits such as being less privacy-invasive compared to using RGB videos, it introduces new challenges which include dealing with low video resolutions and the sensors' sensitiveness towards user motion. We overcome these challenges by diversifying our sign language video dataset to be robust to various usage scenarios via data augmentation and design a set of schemes to emphasize human gestures from the input images for effective sign detection. The inference engine of SUGO is based on a 3-dimensional convolutional neural network (3DCNN) to classify a sequence of video frames as a pre-trained word. Furthermore, the overall operations are designed to be light-weight so that sign language translation takes place in real-time using only the resources available on a smartphone, with no help from cloud servers nor external sensing components. Specifically, to train and test SUGO, we collect sign language data from 20 individuals for 50 Korean Sign Language words, summing up to a dataset of ~5,000 sign gestures and collect additional in-the-wild data to evaluate the performance of SUGO in real-world usage scenarios with different lighting conditions and daily activities. Comprehensively, our extensive evaluations show that SUGO can properly classify sign words with an accuracy of up to 91% and also suggest that the system is suitable (in terms of resource usage, latency, and environmental robustness) to enable a fully mobile solution for sign language translation.

show abstract

“…Currently, the general classifiers include Dynamic Time Warping (DTW), Support Vector Machine (SVM), k-Nearest Neighbor (KNN), Hidden Markov Model (HMM), BP Neural Network (BPNN), Self-Organizing Map (SOM), etc. This article presents some typical behavior recognition applications, including daily behavior recognition [55]- [62], table tennis action recognition [63], bodyweight exercise recognition [64], danger pose detection [65], abnormal activity detection [66], falling detection [67]- [71], hand gesture recognition [72]- [81], sign language recognition [82], [83], sleep monitoring [84], respiration detection [85]- [88], lip reading and speech recognition [89], [90], keystroke detection [91], [92], writing recognition [93]- [95], sedentary behavior monitoring [96], smoking detection [97], crowd counting [98]- [103], step counting [104], [105], human presence detection [106]- [111], and user authentication [112]- [123].…”

Section: A Pattern-based Behavior Recognitionmentioning

confidence: 99%

“…Shang et al propose WiSign [83] and improve it [82] in 2017, two sign language recognition systems that recognize eight sign languages such as Hello, Thanks, Yes, No, etc. The main characteristics of WiSign are that this system leverages training data that solely have sparse labels to recognize gestures by using transfer learning and semi-supervised learning.…”

Section: C: Sign Language Recognitionmentioning

confidence: 99%

A Survey on Human Behavior Recognition Using Channel State Information

et al. 2019

View full text Add to dashboard Cite

Recently, device-free human behavior recognition has become a hot research topic and has achieved significant progress in the field of ubiquitous computing. Among various implementation, behavior recognition based on WiFi CSI (channel state information) has drawn wide attention due to its major advantages. This paper investigates more than 100 latest CSI based behavior recognition applications within the last 6 years and presents a comprehensive survey from every aspect of human behavior recognition. Firstly, this paper reviews general behavior recognition applications using the WiFi signal and presents the basic concept of CSI and the fundamental principle of CSI-based behavior recognition. This paper analyzes the key components and core characteristics of the system architecture of human behavior recognition using CSI. Afterward, we divide the sensing procedures into many steps and summarize the typical studies from these steps, including base signal selection, signal preprocessing, and identification approaches. Next, based on the recognition technique, we classify the applications into three groups, including patternbased, model-based, and deep learning-based approach. In every group, we categorize the state-of-the-art applications into three groups, including coarse-grained specific behavior recognition, fine-grained specific behavior recognition, and activity inference. It elaborates the typical behavior recognition applications from five aspects, including experimental equipment, experimental scenario, behavior, classifier, and system performance. Then, this paper presents comprehensive discussions about representative applications from the implementation view and outlines the major consideration when developing a recognition system. Finally, this article concludes by analyzing the open issues of CSI-based behavior recognition applications and pointing out future research directions. INDEX TERMS Channel state information (CSI), deep learning, human behavior recognition, model, pattern, WiFi.

show abstract

A Robust Sign Language Recognition System with Sparsely Labeled Instances Using Wi-Fi Signals

Cited by 21 publications

References 17 publications

Transfer Learning With Time Series Data: A Systematic Mapping Study

Transfer Learning With Time Series Data: A Systematic Mapping Study

Enabling Real-time Sign Language Translation on Mobile Platforms with On-board Depth Cameras

A Survey on Human Behavior Recognition Using Channel State Information

Contact Info

Product

Resources

About