Higher-Order Attention Networks

Hajij, Mustafa; Papamarkou, Theodore; Miolane, Nina; Guzmán-Sáenz, Aldo; Ramamurthy, Karthikeyan Natesan; Birdal, Tolga; Dey, Tamal K.; Mukherjee, Soham; Samaga, Shreyas N.; Livesay, Neal; Walters, Robin; Rosen, P. A.

doi:10.48550/arxiv.2206.00606

Cited by 6 publications

(5 citation statements)

References 75 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a first numerical test, we compare the performance of our filtering strategy with the approach proposed in [26], adapted to filtering signals defined over cell complexes by simply replacing B 2 with the matrix in (3) capturing the upper incidence relations of a cell complex. Our optimal FIR filters are derived by solving the least squares problem in (32) and (33), where we set α 0 = 0. Indeed, albeit the difference between our approach and that of [26] is minimum, because we simply set α 0 = 0 a priori, while in [26] the choice of α 0 is part of the optimization, we will show that there is a clear advantage in terms of accuracy.…”

Section: Filtering Over Cell Complexesmentioning

confidence: 99%

“…An introduction to the processing of signals over cell complexes has been presented in [31], where it was shown that convolutional filters can serve as building blocks in neural networks defined over cell complexes. Recently, in [32], [33] novel attention-based neural networks have been defined over cell complexes. At a more fundamental level, one of the key unresolved issues in DNN is explainability.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Topological Signal Processing Over Generalized Cell Complexes

Sardellitti,

Barbarossa

2024

IEEE Trans. Signal Process.

View full text Add to dashboard Cite

Topological Signal Processing (TSP) over simplicial complexes is a framework that has been recently proposed, as a generalization of graph signal processing (GSP), to extend GSP to analyzing signals defined over sets of any order (i.e., not only vertices of a graph) and to capture multiway relations of any order among the data. However, simplicial complexes are required to satisfy the so-called inclusion property, according to which, if a set belongs to the complex, then all its subsets must also belong to the complex. In some applications, this is a severe limitation. To overcome this limit, in this paper we extend TSP to deal with signals defined over cell complexes and we also generalize the concept of cell complexes to include hollow cells. We show that, even if the algebraic formulation does not change significantly, the extension to the generalized cell complexes considerably broadens the number of applications. Most important, the new representation provides a much better trade-off between the complexity of the representation and its accuracy. In addition, we propose a method to infer the structure of the cell complex from data and we propose distributed filtering strategies, including a method to retrieve the sparsest representation of the harmonic component. We quantify the advantages of using cell complexes instead of simplicial complexes, in terms of the complexity/accuracy trade-off, for different applications such image segmentation and recovering of real flows measured on data traffic and transportation networks.

show abstract

Section: Filtering Over Cell Complexesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Topological Signal Processing Over Generalized Cell Complexes

Sardellitti,

Barbarossa

2024

IEEE Trans. Signal Process.

View full text Add to dashboard Cite

show abstract

“…x 0 , x 1 , xh, G2: Learned signals and weights (metric tensor) 1: function EDGE FLOW ESTIMATION (Inputs) 2: 19), (20), and (…”

Section: Joint Learning Of Edge Flows and Weightsmentioning

confidence: 99%

“…In [19], weighted persistent homology is proposed to analyse biomolecular data where weights reflect certain physical, chemical or biological properties into the simplicial complex generation. The works in [12,20] proposed attentional deep architectures for simplex-structured data that can be also seen as methods implicitly working on WSCs. Contribution.…”

Section: Introductionmentioning

confidence: 99%

Topological Signal Processing Over Weighted Simplicial Complexes

Battiloro

Sardellitti

Barbarossa

et al. 2023

ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Weighing the topological domain over which data can be represented and analysed is a key strategy in many signal processing and machine learning applications, enabling the extraction and exploitation of meaningful data features and their (higher order) relationships. Our goal in this paper is to present topological signal processing tools for weighted simplicial complexes. Specifically, relying on the weighted Hodge Laplacian theory, we propose efficient strategies to jointly learn the weights of the complex and the filters for the solenoidal, irrotational and harmonic components of the signals defined over the complex. We numerically assess the effectiveness of the proposed procedures.

show abstract

“…These works analyze HOGDMs in the context of complex physical processes [22], [52], [234] and signal processing [213]. A recent work on topological deep learning [122], [123], [192] covers a very broad set of topics related to topological neural networks. Another work [9] focuses on broad representation learning for hypergraphs.…”

Section: Scope Of This Work Vs Related Surveys and Analysesmentioning

confidence: 99%

Motif Prediction with Graph Neural Networks

Besta¹,

Grob²,

Miglioli³

et al. 2022

Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

View full text Add to dashboard Cite

Link prediction is one of the central problems in graph mining. However, recent studies highlight the importance of higher-order network analysis, where complex structures called motifs are the first-class citizens. We first show that existing link prediction schemes fail to effectively predict motifs. To alleviate this, we establish a general motif prediction problem and we propose several heuristics that assess the chances for a specified motif to appear. To make the scores realistic, our heuristics consider -among others -correlations between links, i.e., the potential impact of some arriving links on the appearance of other links in a given motif. Finally, for highest accuracy, we develop a graph neural network (GNN) architecture for motif prediction. Our architecture offers vertex features and sampling schemes that capture the rich structural properties of motifs. While our heuristics are fast and do not need any training, GNNs ensure highest accuracy of predicting motifs, both for dense (e.g., 𝑘-cliques) and for sparse ones (e.g., 𝑘-stars). We consistently outperform the best available competitor by more than 10% on average and up to 32% in area under the curve. Importantly, the advantages of our approach over schemes based on uncorrelated link prediction increase with the increasing motif size and complexity. We also successfully apply our architecture for predicting more arbitrary clusters and communities, illustrating its potential for graph mining beyond motif analysis. CCS CONCEPTS• Information systems → Data mining.

show abstract

Higher-Order Attention Networks

Cited by 6 publications

References 75 publications

Topological Signal Processing Over Generalized Cell Complexes

Topological Signal Processing Over Generalized Cell Complexes

Topological Signal Processing Over Weighted Simplicial Complexes

Motif Prediction with Graph Neural Networks

Contact Info

Product

Resources

About