CycleMLP: A MLP-Like Architecture for Dense Visual Predictions

Chen, Shoufa; Xie, Enze; Ge, Chongjian; Chen, Runjian; Ding, Lixin; Luo, Ping

doi:10.1109/tpami.2023.3303397

Cited by 50 publications

(9 citation statements)

References 62 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [18], a new architecture based on MLP with a very high degree of structured processing was proposed. Its further modifications are reported in [18][19][20][21][22][23][24][25][26]. The disadvantage of this approach is that the number of known architectures that can be used at the first stage of the ensemble classifier is currently limited.…”

Section: Discussion Of Results Of Investigating the Use Of Analogs Of...mentioning

confidence: 99%

See 1 more Smart Citation

Reducing the volume of computations when building analogs of neural networks for the first stage of an ensemble classifier with stacking

Galchonkov,

Baranov,

Chervonenko

et al. 2024

EEJET

View full text Add to dashboard Cite

The object of research in this work is ensemble classifiers with stacking, intended for the classification of objects in images with the presence of small sets of labeled data for training. To improve the quality of classification at the first stage of such a classifier, it is necessary to place more primary classifiers that differ in heterogeneous structured processing. However, the number of known neural networks with appropriate characteristics is limited. One approach to solving this problem is to build analogs of known neural networks that make classification errors on other images compared to the base network. The disadvantage of the known methods for constructing such analogs is the need to perform additional floating-point operations. The current paper proposes and investigates a new method to form analogs through random cyclic shifts of rows or columns of input images. This has made it possible to completely eliminate additional floating-point operations. The effectiveness of using this method is explained by the structured processing of input images in basic neural networks. The use of analogs obtained by the proposed method does not impose additional restrictions in practice. This is because the heterogeneity of structured processing in basic neural networks is a typical requirement for them in an ensemble classifier with stacking. The simulation for the CIFAR-10 data set demonstrated that the proposed technique for constructing analogs allows for a comparative quality of classification by the ensemble classifier. Using MLP-Mixer analogs provided an improvement of 4.6 %, and CCT analogs – 5.9 %

show abstract

Section: Discussion Of Results Of Investigating the Use Of Analogs Of...mentioning

confidence: 99%

“…In [24], the column and row processing structure used in MLP-Mixer is replaced with processing of helical structures with different helix sizes. That made it possible to increase the quality of categorization.…”

Section: The Study Materials and Methodsmentioning

confidence: 99%

Reducing the volume of computations when building analogs of neural networks for the first stage of an ensemble classifier with stacking

Galchonkov,

Baranov,

Chervonenko

et al. 2024

EEJET

View full text Add to dashboard Cite

show abstract

“…The architectural design of MAXIM by Tu et al [40] focused on supporting high-resolution images and considering global properties. CycleMLP, developed by Chen et al [41], was notable for its adaptability and efficiency. Meanwhile, RaftMLP, created by Tatsunami and Taki [42], was distinguished by its novel approach to integrating inductive biases, marking significant advancements in the field.…”

Section: Rethinking Dense Prediction and Image Processing In Mlpsmentioning

confidence: 99%

“…MAXIM's distinctiveness was in its dual focus on global and fully convolutional properties, desir-able in image processing, and its state-of-the-art results on numerous benchmarks with optimized parameters and computational efficiency. CycleMLP is an MLP-like architecture that stands out for its versatility in visual recognition and dense prediction tasks [41]. Unlike contemporary MLP models, CycleMLP operated various image sizes and achieved linear computational complexity relative to image size.…”

Section: Rethinking Dense Prediction and Image Processing In Mlpsmentioning

confidence: 99%

Deciphering the Efficacy of No-Attention Architectures in Computed Tomography Image Classification: A Paradigm Shift

Alzahrani

2024

Mathematics

View full text Add to dashboard Cite

The burgeoning domain of medical imaging has witnessed a paradigm shift with the integration of AI, particularly deep learning, enhancing diagnostic precision and expediting the analysis of Computed Tomography (CT) images. This study introduces an innovative Multilayer Perceptron-driven model, DiagnosticMLP, which sidesteps the computational intensity of attention-based mechanisms, favoring a no-attention architecture that leverages Fourier Transforms for global information capture and spatial gating units for local feature emphasis. This study’s methodology encompasses a sophisticated augmentation and patching strategy at the input level, followed by a series of MLP blocks designed to extract hierarchical features and spatial relationships, culminating in a global average pooling layer before classification. Evaluated against state-of-the-art MLP-based models including MLP-Mixer, FNet, gMLP, and ResMLP across diverse and extensive CT datasets, including abdominal, and chest scans, DiagnosticMLP demonstrated a remarkable ability to converge efficiently, with competitive accuracy, F1 scores, and AUC metrics. Notably, in datasets featuring kidney and abdomen disorders, the model showcased superior generalization capabilities, underpinned by its unique design that addresses the complexity inherent in CT imaging. The findings in terms of accuracy and precision-recall balance posit DiagnosticMLP as an exceptional outperforming alternative to attention-reliant models, paving the way for streamlined, efficient, and scalable AI tools in medical diagnostics, reinforcing the potential for AI-augmented precision medicine without the dependency on attention-based architectures.

show abstract

“…Recently, several deep learning models based on Multi-Layer Perceptron (MLP), including MLP-Mixer [29], ResMLP [30], CycleMLP [31], and S MLP [32], have regained attention in the literature. The pioneering works MLP-Mixer and ResMLP stack two types of MLP layers, namely, token-mixing MLP and channel-mixing MLP, alternately.…”

mentioning

confidence: 99%

A MLP-Mixer and mixture of expert model for remaining useful life prediction of lithium-ion batteries

Zhao,

Song,

Wang

et al. 2023

Front. Comput. Sci.

View full text Add to dashboard Cite

Accurately predicting the Remaining Useful Life (RUL) of lithium-ion batteries is crucial for battery management systems. Deep learning-based methods have been shown to be effective in predicting RUL by leveraging battery capacity time series data. However, the representation learning of features such as long-distance sequence dependencies and mutations in capacity time series still needs to be improved. To address this challenge, this paper proposes a novel deep learning model, the MLP-Mixer and Mixture of Expert (MMMe) model, for RUL prediction. The MMMe model leverages the Gated Recurrent Unit and Multi-Head Attention mechanism to encode the sequential data of battery capacity to capture the temporal features and a re-zero MLP-Mixer model to capture the high-level features. Additionally, we devise an ensemble predictor based on a Mixture-of-Experts (MoE) architecture to generate reliable RUL predictions. The experimental results on public datasets demonstrate that our proposed model significantly outperforms other existing methods, providing more reliable and precise RUL predictions while also accurately tracking the capacity degradation process. Our code and dataset are available at the website of github.

show abstract

CycleMLP: A MLP-Like Architecture for Dense Visual Predictions

Cited by 50 publications

References 62 publications

Reducing the volume of computations when building analogs of neural networks for the first stage of an ensemble classifier with stacking

Reducing the volume of computations when building analogs of neural networks for the first stage of an ensemble classifier with stacking

Deciphering the Efficacy of No-Attention Architectures in Computed Tomography Image Classification: A Paradigm Shift

A MLP-Mixer and mixture of expert model for remaining useful life prediction of lithium-ion batteries

Contact Info

Product

Resources

About