Electronic eye and electronic tongue data fusion combined with a GETNet model for the traceability and detection of Astragalus

Abstract

BackgroundAstragalus is a widely used traditional Chinese medicine material that is easily confused due to its quality, price and other factors derived from different origins. This article describes a novel method for the rapid tracing and detection of Astragalus via the joint application of an electronic tongue (ET) and an electronic eye (EE) combined with a lightweight CNN‐transformer model. First, ET and EE systems are employed to measure the taste fingerprints and appearance images of different Astragalus samples, respectively. Three spectral transform methods, the Markov transition field (MTF), short‐time Fourier transform (STFT) and recurrence plot (RP), are utilized to convert the ET signals into 2D spectrograms. Then, the obtained ET spectrograms are fused with the EE image to obtain multimodal information. A lightweight hybrid model, termed GETNet, is designed to achieve pattern recognition for the Astragalus fusion information. The proposed model employs an improved transformer module and an improved Ghost bottleneck as its backbone network, complementarily utilizing the benefits of CNN and transformer architectures for local and global feature representation. Furthermore, the Ghost bottleneck is further optimized using a channel attention technique, which boosts the model's feature extraction effectiveness.ResultsThe experiments indicate that the proposed data fusion strategy based on ET and EE devices has better recognition accuracy than that attained with independent sensing devices.ConclusionThe proposed method achieves a high precision (99.1%) and recall (99.1%) values, providing a novel approach for rapidly identifying the origin of Astragalus, and it holds great promise for applications involving other types of Chinese herbal medicines.This article is protected by copyright. All rights reserved.