Do We Really Need Complicated Model Architectures For Temporal Networks?

Abstract

Recurrent neural network (RNN) and self-attention mechanism (SAM) are the de facto methods to extract spatial-temporal information for temporal graph learning. Interestingly, we found that although both RNN and SAM could lead to a good performance, in practice neither of them is always necessary. In this paper, we propose GraphMixer, a conceptually and technically simple architecture that consists of three components: 1 a link-encoder that is only based on multi-layer perceptrons (MLP) to summarize the information from temporal links, 2 a node-encoder that is only based on neighbor mean-pooling to summarize node information, and 3 an MLP-based link classifier that performs link prediction based on the outputs of the encoders. Despite its simplicity, GraphMixer attains an outstanding performance on temporal link prediction benchmarks with faster convergence and better generalization performance. These results motivate us to rethink the importance of simpler model architecture. [Code].

iLoRE: Dynamic Graph Representation with Instant Long-term Modeling and Re-occurrence Preservation

“…Subsequently, many graph models emerged such as [4,19,22,29,31,36,38], further increasing the popularity of dynamic graph modeling. However, recent research has found that the "graph module" in graph models is not necessary [6,28]. In [6], the authors simply use a Multi-Layer Procedure (MLP) to model one-hop historical neighbors' information and achieve best results than previous graph models, causing researchers to reconsider the necessity of graph modules.…”

“…For evaluation, we choose nine dynamic graph modeling methods to compare with ours, including CTDNE [17], DyRep [23], JODIE [11], TGAT [33], TGN [19], CAW [31], TIGER [36], GraMixer [6], PINT [22]. Note that CTDNE can not be applied in the inductive setting, and CAW can not be conducted in the evolving node classification task.…”

“…This observation proves the excellent effectiveness and expressiveness of our method. For all baselines, existing sequential models, i.e., [6,11,17,23], perform worse than graph models. This may be owing to the fact that graph models, whose nodes can attend to multi-hop neighbors, have preserved the longer neighbors' information during training.…”

A Study on the Curriculum of the Graduate School of Department of Education Consulting in South Korea

“…In addition to these, various studies [24,15,16,17] have been conducted on continuous-time TGNNs. Most research on continuous-time TGNNs has demonstrated their advancement using link prediction as a downstream task, so we mainly investigate link prediction as our target task to study the effect of adversarial attacks.…”

“…TGNNs jointly learn the temporal, structural, and contextual relationships present in CTDGs by encoding graph information into time-aware node embeddings. By incorporating temporal information, TGNNs have demonstrated superior performance over static GNNs in various tasks like link prediction [17,18] and dynamic node classification [12]. Nonetheless, in contrast to the actively researched adversarial attacks on static GNNs, the vulnerabilities of TGNNs to adversarial attacks remain underexplored, yet the significance of conducting such research is undeniable.…”