SNoRe: Scalable Unsupervised Learning of Symbolic Node Representations

Mežnar, Sebastian; Lavrač, Nada; Škrlj, Blaž

doi:10.1109/access.2020.3039541

Cited by 7 publications

(10 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, as ReliefE requires merely the representations of instances (training or target), the proposed approach generalizes well beyond tabular data with a single adaptation: the embedding method needs to be suitable for the considered data type. For example, if an instance is described by an ordered list of graphs, the plethora of graph embedding methods (Goyal and Ferrara 2018;Mežnar et al 2020) could be used to prioritize the graphs based on their (learned) representations. Similarly, ReliefE could be adapted for learning in the context of relational data bases, via Wordification (Perovšek et al 2015) and other propositionalization-like algorithms.…”

Section: How Powerful Is Reliefe?mentioning

confidence: 99%

ReliefE: feature ranking in high-dimensional spaces via manifold embeddings

et al. 2021

View full text Add to dashboard Cite

Feature ranking has been widely adopted in machine learning applications such as high-throughput biology and social sciences. The approaches of the popular Relief family of algorithms assign importances to features by iteratively accounting for nearest relevant and irrelevant instances. Despite their high utility, these algorithms can be computationally expensive and not-well suited for high-dimensional sparse input spaces. In contrast, recent embedding-based methods learn compact, low-dimensional representations, potentially facilitating down-stream learning capabilities of conventional learners. This paper explores how the Relief branch of algorithms can be adapted to benefit from (Riemannian) manifold-based embeddings of instance and target spaces, where a given embedding’s dimensionality is intrinsic to the dimensionality of the considered data set. The developed ReliefE algorithm is faster and can result in better feature rankings, as shown by our evaluation on 20 real-life data sets for multi-class and multi-label classification tasks. The utility of ReliefE for high-dimensional data sets is ensured by its implementation that utilizes sparse matrix algebraic operations. Finally, the relation of ReliefE to other ranking algorithms is studied via the Fuzzy Jaccard Index.

show abstract

Section: How Powerful Is Reliefe?mentioning

confidence: 99%

ReliefE: feature ranking in high-dimensional spaces via manifold embeddings

et al. 2021

View full text Add to dashboard Cite

show abstract

“…However, as ReliefE requires merely the representations of instances (training or target), the proposed approach generalizes well beyond tabular data with a single adaptation: the embedding method needs to be suitable for the considered data type. For example, if an instance is described by an ordered list of graphs, the plethora of graph embedding methods [17,34] could be used to prioritize the graphs based on their (learned) representations. Similarly, ReliefE could be adapted for learning in the context of relational data bases, via Wordification [38] and other propositionalization-like algorithms.…”

Section: How Powerful Is Reliefe?mentioning

confidence: 99%

ReliefE: Feature Ranking in High-dimensional Spaces via Manifold Embeddings

Škrlj¹,

Džeroski²,

Lavrač³

et al. 2021

Preprint

View full text Add to dashboard Cite

Feature ranking has been widely adopted in machine learning applications such as high-throughput biology and social sciences. The approaches of the popular Relief family of algorithms assign importances to features by iteratively accounting for nearest relevant and irrelevant instances. Despite their high utility, these algorithms can be computationally expensive and not-well suited for high-dimensional sparse input spaces. In contrast, recent embeddingbased methods learn compact, low-dimensional representations, potentially facilitating down-stream learning capabilities of conventional learners. This paper explores how the Relief branch of algorithms can be adapted to benefit from (Riemannian) manifold-based embeddings of instance and target spaces, where a given embedding's dimensionality is intrinsic to the dimensionality of the considered data set. The developed ReliefE algorithm is faster and can result in better feature rankings, as shown by our evaluation on 20 real-life data sets for multi-class and multi-label classification tasks. The utility of ReliefE for high-dimensional data sets is ensured by its implementation that utilizes sparse matrix algebraic operations. Finally, the relation of ReliefE to other ranking algorithms is studied via the Fuzzy Jaccard Index.

show abstract

“…Another group of approaches embed graphs into tabular data which is used together with traditional machine learning methods such as logistic regression to generate predictions. These approaches include well-established and tested methods such as node2vec [27] and Deepwalk [48], as well as some new ones such as SNoRe [41]. Recently, with new research in deep learning approaches, neural network models such as graph convolutional networks (GCN) [33], and graph attention networks (GAT) [62] have emerged as end-to-end learners.…”

Section: Graph-based Machine Learningmentioning

confidence: 99%

“…Another approach, SNoRe [41], creates a sparse embedding where a node is represented as a vector of similarities between the hashed neighbourhood of the node and the neighbourhoods of nodes selected as features.…”

Section: Link Predictionmentioning

confidence: 99%

See 1 more Smart Citation

Link Analysis meets Ontologies: Are Embeddings the Answer?

Mežnar¹,

Bevec²,

Lavrač³

et al. 2021

Preprint

View full text Add to dashboard Cite

The increasing amounts of semantic resources offer valuable storage of human knowledge; however, the probability of wrong entries increases with the increased size. The development of approaches that identify potentially spurious parts of a given knowledge base is thus becoming an increasingly important area of interest. In this work, we present a systematic evaluation of whether structure-only link analysis methods can already offer a scalable means to detecting possible anomalies, as well as potentially interesting novel relation candidates. Evaluating thirteen methods on eight different semantic resources, including Gene Ontology, Food Ontology, Marine Ontology and similar, we demonstrated that structure-only link analysis could offer scalable anomaly detection for a subset of the data sets. Further, we demonstrated that by considering symbolic node embedding, explanations of the predictions (links) could be obtained, making this branch of methods potentially more valuable than the black-box only ones. To our knowledge, this is currently one of the most extensive systematic studies of the applicability of different types of link analysis methods across semantic resources from different domains.

show abstract

SNoRe: Scalable Unsupervised Learning of Symbolic Node Representations

Cited by 7 publications

References 25 publications

ReliefE: feature ranking in high-dimensional spaces via manifold embeddings

ReliefE: feature ranking in high-dimensional spaces via manifold embeddings

ReliefE: Feature Ranking in High-dimensional Spaces via Manifold Embeddings

Link Analysis meets Ontologies: Are Embeddings the Answer?

Contact Info

Product

Resources

About