Automatic Thai Finger Spelling Transcription

Abstract

This article explores a transcription of a video recording Thai Finger Spelling (TFS)—a specific signing mode used in Thai sign language—to a corresponding Thai word. TFS copes with 42 Thai alphabets and 20 vowels using multiple and complex schemes. This leads to many technical challenges uncommon in spelling schemes of other sign languages. Our proposed system, Automatic Thai Finger Spelling Transcription (ATFS), processes a signing video in 3 stages: ALS marking video frames to easily remove any non-signing frame as well as conveniently group frames associating to the same alphabet, SR classifying a signing image frame to a sign label (or its equivalence), and SSR transcribing a series of signs into alphabets. ALS utilizes the TFS practice of signing different alphabets at different locations. SR and SSC employ well-adopted spatial and sequential models. Our ATFS has been found to achieve Alphabet Error Rate (AER) 0.256 (c.f. 0.63 of the baseline method). In addition to ATFS, our findings have disclosed a benefit of coupling image classification and sequence modeling stages by using a feature or penultimate vector for label representation rather than a definitive label or one-hot coding. Our results also assert the necessity of a smoothening mechanism in ALS and reveal a benefit of our proposed WFS, which could lead to over 15.88 % improvement. For TFS transcription, our work emphasizes the utilization of signing location in the identification of different alphabets. This is contrary to a common belief of exploiting signing time duration, which are shown to be ineffective by our data. HIGHLIGHTS Prototype of Thai finger spelling transcription (transcribing a signing video to alphabets) Utilization of signing location as cue for identification of different alphabets Disclosure of a benefit of coupling image classification and sequence modeling in signing transcription Examination of various frame smoothing techniques and their contributions to the overall transcription performance GRAPHICAL ABSTRACT

Vacancy and Growth Modulation of Cobalt Hexacyanoferrate by Porous MXene for Zinc Ion Batteries

“…Fe K-edge XANES spectra of the samples were also investigated to analyze changes in the Fe atomic conditions induced by composition with the MXene nanosheet (Figure S8, Supporting Information). In alignment with the XPS results, the E 0 of the three samples shows minimal variations as they are all located at almost the identical position of 7126.6 eV, which matches precisely with the E 0 of the Fe 2 O 3 , [43] confirming the 3+ Fe average oxidation state for the bulk of the CoHCF crystals. Moreover, the composite preedge peaks at 7116.5 eV are found to be less intensive and less distinctive when compared to the pure CoHCF counterpart.…”

TFS Recognition: Investigating MPH]{Thai Finger Spelling Recognition: Investigating MediaPipe Hands Potentials

“…Figure 2 Designated areas of TFS signs in the static-point-on-hand scheme TFS sign recognition has gained some research attention [4]. Most previous works [4][5][6][7] rely on training recognition models from scratch.…”

“…Figure 2 Designated areas of TFS signs in the static-point-on-hand scheme TFS sign recognition has gained some research attention [4]. Most previous works [4][5][6][7] rely on training recognition models from scratch. However, Medi-aPipe Hands (MPH), high precision hand-landmark localization provided by Google [8], has recently been made publicly available.…”