LW-ViT: The Lightweight Vision Transformer Model Applied in Offline Handwritten Chinese Character Recognition

Geng, Shiyong; Zhu, Zongnan; Wang, Zhida; Dan, Yongping; Li, Hengyi

doi:10.3390/electronics12071693

Cited by 6 publications

(1 citation statement)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These methods inherit the flexibility of transformers in modeling long-range dependencies and can maintain long-distance spatial information and dependencies of different feature channels. Large receptive fields are crucial for the success of transformer-based models [40][41][42], especially for fractured and fallen markings. This study proposes a new dynamic multiscale large selection kernel network (LSKNet), which dynamically adjusts the receptive field of the feature extraction backbone to improve segmentation accuracy and robustness.…”

Section: Transformer-based Methodsmentioning

confidence: 99%

M-SKSNet: Multi-Scale Spatial Kernel Selection for Image Segmentation of Damaged Road Markings

Wang,

Liao,

Wang

et al. 2024

Remote Sensing

View full text Add to dashboard Cite

It is a challenging task to accurately segment damaged road markings from images, mainly due to their fragmented, dense, small-scale, and blurry nature. This study proposes a multi-scale spatial kernel selection net named M-SKSNet, a novel model that integrates a transformer and a multi-dilated large kernel convolutional neural network (MLKC) block to address these issues. Through integrating multiple scales of information, the model can extract high-quality and semantically rich features while generating damage-specific representations. This is achieved by leveraging both the local and global contexts, as well as self-attention mechanisms. The performance of M-SKSNet is evaluated both quantitatively and qualitatively, and the results show that M-SKSNet achieved the highest improvement in F1 by 3.77% and in IOU by 4.6%, when compared to existing models. Additionally, the effectiveness of M-SKSNet in accurately extracting damaged road markings from images in various complex scenarios (including city roads and highways) is demonstrated. Furthermore, M-SKSNet is found to outperform existing alternatives in terms of both robustness and accuracy.

show abstract

Section: Transformer-based Methodsmentioning

confidence: 99%

M-SKSNet: Multi-Scale Spatial Kernel Selection for Image Segmentation of Damaged Road Markings

Wang,

Liao,

Wang

et al. 2024

Remote Sensing

View full text Add to dashboard Cite

show abstract

Application of the transformer model algorithm in chinese word sense disambiguation: a case study in chinese language

Li,

Wang

et al. 2024

Sci Rep

View full text Add to dashboard Cite

This study aims to explore the research methodology of applying the Transformer model algorithm to Chinese word sense disambiguation, seeking to resolve word sense ambiguity in the Chinese language. The study introduces deep learning and designs a Chinese word sense disambiguation model based on the fusion of the Transformer with the Bi-directional Long Short-Term Memory (BiLSTM) algorithm. By utilizing the self-attention mechanism of Transformer and the sequence modeling capability of BiLSTM, this model efficiently captures semantic information and context relationships in Chinese sentences, leading to accurate word sense disambiguation. The model’s evaluation is conducted using the PKU Paraphrase Bank, a Chinese text paraphrase dataset. The results demonstrate that the model achieves a precision rate of 83.71% in Chinese word sense disambiguation, significantly outperforming the Long Short-Term Memory algorithm. Additionally, the root mean squared error of this algorithm is less than 17, with a loss function value remaining around 0.14. Thus, this study validates that the constructed Transformer-fused BiLSTM-based Chinese word sense disambiguation model algorithm exhibits both high accuracy and robustness in identifying word senses in the Chinese language. The findings of this study provide valuable insights for advancing the intelligent development of word senses in Chinese language applications.

show abstract

GNSS location error reduction method for microtremor survey system based on EMD-CNN-LSTM

Wu,

Mou

2024

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

Large errors exist when the microtremor survey system uses the global navigation satellite system (GNSS) for static localization. Aiming at the problem that the existing methods cannot effectively weaken the random error and multipath error, an error weakening method based on Empirical Mode Decomposition (EMD), Convolutional Neural Networks (CNN), and Long Short-term Memory Networks (LSTM) is proposed. The model first uses EMD to decompose the high-frequency random error, then reconstructs the low-frequency component and extracts the local features using CNN, and finally learns the change rule of multipath error using LSTM and weakens it. The model can remove random errors in the early stage while reducing the interference of noise on the neural network in the later stage and then improve the accuracy of localization. The experimental results show that the model can effectively improve the localization accuracy in the case of short-time measurements so that the localization accuracy in the E, N, and U directions can be improved by 74.57%, 74.76%, and 71.86%, respectively, which is more than 10% higher than the localization accuracy improvement rate of the existing CNN-LSTM model.

show abstract

LW-ViT: The Lightweight Vision Transformer Model Applied in Offline Handwritten Chinese Character Recognition

Cited by 6 publications

References 48 publications

M-SKSNet: Multi-Scale Spatial Kernel Selection for Image Segmentation of Damaged Road Markings

M-SKSNet: Multi-Scale Spatial Kernel Selection for Image Segmentation of Damaged Road Markings

Application of the transformer model algorithm in chinese word sense disambiguation: a case study in chinese language

GNSS location error reduction method for microtremor survey system based on EMD-CNN-LSTM

Contact Info

Product

Resources

About