A multi‐modal clustering method for traditional Chinese medicine clinical data via media convergence

Abstract

Media convergence is a media change led by technological innovation. Applying media convergence technology to the study of clustering in Chinese medicine can significantly exploit the advantages of media fusion. Obtaining consistent and complementary information among multiple modalities through media convergence can provide technical support for clustering. This article presents an approach based on Media Convergence and Graph convolution Encoder Clustering (MCGEC) for traditonal Chinese medicine (TCM) clinical data. It feeds modal information and graph structure from media information into a multi-modal graph convolution encoder to obtain the media feature representation learnt from multiple modalities. MCGEC captures latent information from various modalities by fusion and optimises the feature representations and network architecture with learnt clustering labels. The experiment is conducted on real-world multimodal TCM clinical data, including information like images and text. MCGEC has improved clustering results compared to the generic single-modal clustering methods and the current more advanced multi-modal clustering methods. MCGEC applied to TCM clinical datasets can achieve better results. Integrating multimedia features into clustering algorithms offers significant benefits compared to single-modal clustering approaches that simply concatenate features from different modalities. It provides practical technical support for multi-modal clustering in the TCM field incorporating multimedia features.

Optimising Urban Freight Logistics Using Discrete-Event Simulation and Cluster Analysis: A Stochastic Two-Tier Hub-and-Spoke Architecture Approach

“…To analyse a group of homogeneous locations and simplify the network, clustering methods are used to partition and differentiate locations. The clustering method has been used in different disciplines to group data points such as biology [37], medicine [38], chemistry [39], and computer science [40].…”

Guest Editorial: Special issue on media convergence and intelligent technology in the metaverse

“…The 10th article presents an approach based on Media Convergence and Graph convolution Encoder Clustering (MCGEC) for TCM clinical data [10]. It feeds modal information and graph structure from media information into a multi-modal graph convolution encoder to obtain the media feature representation learnt from multiple modalities.…”