Robust Training of Graph Neural Networks via Noise Governance

Qian, Siyi; Ying, Haochao; Hu, Renjun; Zhou, Jingbo; Chen, Jintai; Chen, Danny Z.; Wu, Jian

doi:10.1145/3539597.3570369

Cited by 10 publications

(1 citation statement)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The strategic use of prompt tokens in conjunction with the top-k selection also allows AMFormer to disregard immaterial correlations in the data. This not only helps in preventing overfitting but also improves the model's resilience against data noise (Cheng et al 2023;Qian et al 2023).…”

Section: Optimization With Prompt Tokensmentioning

confidence: 99%

Arithmetic Feature Interaction Is Necessary for Deep Tabular Learning

Cheng,

Hu,

Ying

et al. 2024

AAAI

View full text Add to dashboard Cite

Until recently, the question of the effective inductive bias of deep models on tabular data has remained unanswered. This paper investigates the hypothesis that arithmetic feature interaction is necessary for deep tabular learning. To test this point, we create a synthetic tabular dataset with a mild feature interaction assumption and examine a modified transformer architecture enabling arithmetical feature interactions, referred to as AMFormer. Results show that AMFormer outperforms strong counterparts in fine-grained tabular data modeling, data efficiency in training, and generalization. This is attributed to its parallel additive and multiplicative attention operators and prompt-based optimization, which facilitate the separation of tabular samples in an extended space with arithmetically-engineered features. Our extensive experiments on real-world data also validate the consistent effectiveness, efficiency, and rationale of AMFormer, suggesting it has established a strong inductive bias for deep learning on tabular data. Code is available at https://github.com/aigc-apps/AMFormer.

show abstract

Section: Optimization With Prompt Tokensmentioning

confidence: 99%