RAFF-Net: An improved tongue segmentation algorithm based on residual attention network and multiscale feature fusion

Song, Haibei; Huang, Zonghai; Li, Feng; Zhong, Yanmei; Wen, Chuanbiao; Guo, Jinhong

doi:10.1177/20552076221136362

Cited by 6 publications

(4 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By sharing parameters, they simultaneously completed the tasks of tongue image segmentation and tongue image classification. Song et al [40] optimized the encoder of the UNet network by fusing the squeeze-and-excitation (SE) blocks between the residual blocks of ResNet, enhancing the feature extraction capability of the encoder. Then, they completed the optimization by introducing a weighted cross-entropy loss function.…”

Section: Deep Learning Segmentationmentioning

confidence: 99%

S5Utis: Structured State-Space Sequence SegNeXt UNet-like Tongue Image Segmentation in Traditional Chinese Medicine

Song,

Zhang,

Shi

et al. 2024

Sensors

View full text Add to dashboard Cite

Intelligent Traditional Chinese Medicine can provide people with a convenient way to participate in daily health care. The ease of acceptance of Traditional Chinese Medicine is also a major advantage in promoting health management. In Traditional Chinese Medicine, tongue imaging is an important step in the examination process. The segmentation and processing of the tongue image directly affects the results of intelligent Traditional Chinese Medicine diagnosis. As intelligent Traditional Chinese Medicine continues to develop, remote diagnosis and patient participation will play important roles. Smartphone sensor cameras can provide irreplaceable data collection capabilities in enhancing interaction in smart Traditional Chinese Medicine. However, these factors lead to differences in the size and quality of the captured images due to factors such as differences in shooting equipment, professionalism of the photographer, and the subject’s cooperation. Most current tongue image segmentation algorithms are based on data collected by professional tongue diagnosis instruments in standard environments, and are not able to demonstrate the tongue image segmentation effect in complex environments. Therefore, we propose a segmentation algorithm for tongue images collected in complex multi-device and multi-user environments. We use convolutional attention and extend state space models to the 2D environment in the encoder. Then, cross-layer connection fusion is used in the decoder part to fuse shallow texture and deep semantic features. Through segmentation experiments on tongue image datasets collected by patients and doctors in real-world settings, our algorithm significantly improves segmentation performance and accuracy.

show abstract

Section: Deep Learning Segmentationmentioning

confidence: 99%

S5Utis: Structured State-Space Sequence SegNeXt UNet-like Tongue Image Segmentation in Traditional Chinese Medicine

Song,

Zhang,

Shi

et al. 2024

Sensors

View full text Add to dashboard Cite

show abstract

“…Therefore, in recent years, many scholars have conducted research on tongue image segmentation based on deep learning models. The research on tongue image segmentation is mainly divided into fine-tuning, optimization, and improvement based on the classical semantic segmentation model UNet, 4 – 7 which can achieve good tongue image segmentation effect; Some researchers also optimized and improved DeepLabV3 or DeepLabV3+ models 8 in order to obtain better tongue image segmentation effect. To improve the segmentation precision and accuracy of the model, the attention mechanism is introduced to solve the problem of inaccurate tongue segmentation and edge segmentation, and the edge information is filtered by the attention feature map, and important features are selected.…”

Section: Introductionmentioning

confidence: 99%

RTC_TongueNet: An improved tongue image segmentation model based on DeepLabV3

Tang,

Tan,

et al. 2024

DIGITAL HEALTH

View full text Add to dashboard Cite

Objective Tongue segmentation as a basis for automated tongue recognition studies in Chinese medicine, which has defects such as network degradation and inability to obtain global features, which seriously affects the segmentation effect. This article proposes an improved model RTC_TongueNet based on DeepLabV3, which combines the improved residual structure and transformer and integrates the ECA (Efficient Channel Attention Module) attention mechanism of multiscale atrous convolution to improve the effect of tongue image segmentation. Methods In this paper, we improve the backbone network based on DeepLabV3 by incorporating the transformer structure and an improved residual structure. The residual module is divided into two structures and uses different residual structures under different conditions to speed up the frequency of shallow information mapping to deep network, which can more effectively extract the underlying features of tongue image; introduces ECA attention mechanism after concat operation in ASPP (Atrous Spatial Pyramid Pooling) structure to strengthen information interaction and fusion, effectively extract local and global features, and enable the model to focus more on difficult-to-separate areas such as tongue edge, to obtain better segmentation effect. Results The RTC_TongueNet network model was compared with FCN (Fully Convolutional Networks), UNet, LRASPP (Lite Reduced ASPP), and DeepLabV3 models on two datasets. On the two datasets, the MIOU (Mean Intersection over Union) and MPA (Mean Pixel Accuracy) values of the classic model DeepLabV3 were higher than those of FCN, UNet, and LRASPP models, and the performance was better. Compared with the DeepLabV3 model, the RTC_TongueNet network model increased MIOU value by 0.9% and MPA value by 0.3% on the first dataset; MIOU increased by 1.0% and MPA increased by 1.1% on the second dataset. RTC_TongueNet model performed best on both datasets. Conclusion In this study, based on DeepLabV3, we apply the improved residual structure and transformer as a backbone to fully extract image features locally and globally. The ECA attention module is combined to enhance channel attention, strengthen useful information, and weaken the interference of useless information. RTC_TongueNet model can effectively segment tongue images. This study has practical application value and reference value for tongue image segmentation.

show abstract

“…Ruan et al constructed an efficient tongue image segmentation model by optimizing the UNet network and designed a new network to specifically handle tongue edge segmentation [9]. Haibei Song et al proposed RAFF-NET for tongue region segmentation [10]。 The above-mentioned study achieved good results, but did not investigate tongue fissures. Existing methods that combine deep learning and tongue diagnosis primarily fall into two categories: object detection [11] and instance segmentation.…”

Section: Introductionmentioning

confidence: 99%

“…For example, Yushan Xue et al used crack and non-crack regions to train AlexNet, extracting deep features of the crack region, and finally performed classification using Support Vector Machines (SVM) [9]. Jianjun Yan et al proposed the Segmentation-Based Deep Learning (SBDL) model for cracked tongue image extraction and recognition [10]. Meng-Yi Li et al improved the partial encoder of the Unet architecture by introducing a global convolutional network module to address the encoder's inability to extract relatively abstract high-level semantic features, thereby achieving cracked tongue extraction.…”

Section: Introductionmentioning

confidence: 99%

Cracked Tongue Extraction Model Based on Improved U-Net Method

Zhang,

Zheng,

Zhao

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Tongue diagnosis holds significant importance in Traditional Chinese Medicine (TCM), with cracked tongues serving as a key diagnostic feature. However, the considerable variability in the morphology, depth, and distribution of tongue cracks poses a challenge for accurate extraction. In this paper, a novel deep learning approach is proposed to enhance the decoder of the U-Net model for cracked tongue extraction by incorporating the Hybrid Parallel Attention Mechanism (HPAM). The inclusion of HPAM enables the model to better concentrate on the small-scale feature information of tongue cracks, thereby improving the accuracy of crack segmentation. Experimental results demonstrate the effectiveness of the proposed method across all three tongue crack datasets. The method achieves a MIoU of 69.31% on the open environment dataset, 76.05% MIoU on the non-open environment dataset, and an overall MIoU of 76.92% on the combined dataset. These results signify a significant improvement over existing methods. This study not only offers an effective approach for automating the extraction of cracked tongues but also contributes to the automation and accuracy of tongue diagnosis, thereby benefiting the field of TCM.INDEX TERMS Cracked tongue, deep learning, attention mechanism.

show abstract

RAFF-Net: An improved tongue segmentation algorithm based on residual attention network and multiscale feature fusion

Cited by 6 publications

References 37 publications

S5Utis: Structured State-Space Sequence SegNeXt UNet-like Tongue Image Segmentation in Traditional Chinese Medicine

S5Utis: Structured State-Space Sequence SegNeXt UNet-like Tongue Image Segmentation in Traditional Chinese Medicine

RTC_TongueNet: An improved tongue image segmentation model based on DeepLabV3

Cracked Tongue Extraction Model Based on Improved U-Net Method

Contact Info

Product

Resources

About