Safe Triplet Screening for Distance Metric Learning

Yoshida, Takaichi; Takeuchi, Ichiro; Karasuyama, Masayuki

doi:10.1145/3219819.3220037

Cited by 10 publications

(8 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similar bounds to those of which we derived here were previously considered for the triplet screening of metric learning on usual numerical data (Yoshida et al, 2018(Yoshida et al, , 2019b. Here, we extend a similar idea to derive subgraph screening.…”

Section: Sphere Bound For Optimal Solutionsupporting

confidence: 64%

Distance Metric Learning for Graph Structured Data

Yoshida,

Takeuchi,

Karasuyama

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Graphs are versatile tools for representing structured data. Therefore, a variety of machine learning methods have been studied for graph data analysis. Although many of those learning methods depend on the measurement of differences between input graphs, defining an appropriate distance metric for a graph remains a controversial issue. Hence, we propose a supervised distance metric learning method for the graph classification problem. Our method, named interpretable graph metric learning (IGML), learns discriminative metrics in a subgraph-based feature space, which has a strong graph representation capability. By introducing a sparsity-inducing penalty on a weight of each subgraph, IGML can identify a small number of important subgraphs that can provide insight about the given classification task. Since our formulation has a large number of optimization variables, an efficient algorithm is also proposed by using pruning techniques based on safe screening and working set selection methods. An important property of IGML is that the optimality of the solution is guaranteed because the problem is formulated as a convex problem and our pruning strategies only discard unnecessary subgraphs. Further, we show that IGML is also applicable to other structured data such as item-set and sequence data, and that it can incorporate vertex-label similarity by using a transportation-based subgraph feature. We empirically evaluate the computational efficiency and classification performance on several benchmark datasets and show some illustrative examples demonstrating that IGML identifies important subgraphs from a given graph dataset.

show abstract

Section: Sphere Bound For Optimal Solutionsupporting

confidence: 64%

Distance Metric Learning for Graph Structured Data

Yoshida,

Takeuchi,

Karasuyama

2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Similar bounds to those derived here were previously considered for the triplet screening of metric learning on usual numerical data (Yoshida et al 2018(Yoshida et al , 2019b. Here, we extend a similar idea to derive subgraph screening.…”

Section: Sphere Bound For Optimal Solutionsupporting

confidence: 63%

Distance metric learning for graph structured data

2021

Self Cite

View full text Add to dashboard Cite

Graphs are versatile tools for representing structured data. As a result, a variety of machine learning methods have been studied for graph data analysis. Although many such learning methods depend on the measurement of differences between input graphs, defining an appropriate distance metric for graphs remains a controversial issue. Hence, we propose a supervised distance metric learning method for the graph classification problem. Our method, named interpretable graph metric learning (IGML), learns discriminative metrics in a subgraph-based feature space, which has a strong graph representation capability. By introducing a sparsity-inducing penalty on the weight of each subgraph, IGML can identify a small number of important subgraphs that can provide insight into the given classification task. Because our formulation has a large number of optimization variables, an efficient algorithm that uses pruning techniques based on safe screening and working set selection methods is also proposed. An important property of IGML is that solution optimality is guaranteed because the problem is formulated as a convex problem and our pruning strategies only discard unnecessary subgraphs. Furthermore, we show that IGML is also applicable to other structured data such as itemset and sequence data, and that it can incorporate vertex-label similarity by using a transportation-based subgraph feature. We empirically evaluate the computational efficiency and classification performance of IGML on several benchmark datasets and provide some illustrative examples of how IGML identifies important subgraphs from a given graph dataset.

show abstract

“…As opposed to correlationbased feature selection techniques (Fan and Lv, 2008;Tibshirani et al, 2011), safe screening strategies are guaranteed to remove only coordinates that do no belong to the solution support. Beyond Lasso, safe screening has also been used for other machine learning problems, such as: binary logistic regression (El Ghaoui et al, 2012;Ndiaye et al, 2017), metric learning (Yoshida et al, 2018), nuclear norm minimization (Zhou and Zhao, 2015) and support vector machine (Ogawa et al, 2013;Zimmert et al, 2015).…”

Section: Introductionmentioning

confidence: 99%

Expanding boundaries of Gap Safe screening

Dantas,

Soubies,

Févotte

2021

Preprint

View full text Add to dashboard Cite

Sparse optimization problems are ubiquitous in many fields such as statistics, signal/image processing and machine learning. This has led to the birth of many iterative algorithms to solve them. A powerful strategy to boost the performance of these algorithms is known as safe screening: it allows the early identification of zero coordinates in the solution, which can then be eliminated to reduce the problem's size and accelerate convergence. In this work, we extend the existing Gap Safe screening framework by relaxing the global strong-concavity assumption on the dual cost function. Instead, we exploit local regularity properties, that is, strong concavity on well-chosen subsets of the domain. The non-negativity constraint is also integrated to the existing framework. Besides making safe screening possible to a broader class of functions that includes β-divergences (e.g., the Kullback-Leibler divergence), the proposed approach also improves upon the existing Gap Safe screening rules on previously applicable cases (e.g., logistic regression). The proposed general framework is exemplified by some notable particular cases: logistic function, β = 1.5 and Kullback-Leibler divergences. Finally, we showcase the effectiveness of the proposed screening rules with different solvers (coordinate descent, multiplicativeupdate and proximal gradient algorithms) and different data sets (binary classification, hyperspectral and count data).

show abstract

Safe Triplet Screening for Distance Metric Learning

Cited by 10 publications

References 17 publications

Distance Metric Learning for Graph Structured Data

Distance Metric Learning for Graph Structured Data

Distance metric learning for graph structured data

Expanding boundaries of Gap Safe screening

Contact Info

Product

Resources

About