Deep Graph Matching via Blackbox Differentiation of Combinatorial Solvers

Deep Probabilistic Graph Matching

“…Zanfir et al [17] formulate graph matching as a quadratic assignment problem under both unary and pairwise affinities, and adopt a spectral matching algorithm [9] as the combinatorial solver that drops both discrete and one-to-one matching constraints in optimization. Rolínek et al [22] relax the graph matching problem based on Lagrangian decomposition, which is solved by embedding BlackBox implementations of a heavily optimized solver [38] based on dual block coordinate ascent. Reformulating graph matching as Koopmans-Beckmann's QAP [39] to minimize the adjacency discrepancy of graphs to be matched, Gao et al [20] adopt the Frank-Wolf algorithm [40] to obtain approximate solutions.…”

“…In [17], the unary and pair-wise affinities are generated by learning deep node and edge representations, and the quadratic assignment problem is solved by a relaxation manner, i.e., spectral matching [9]. A more advanced method has been proposed by Rolínek et al [22] who propose using strong feature extraction with SplineCNN [42] and leverage the combinatorial solver based on dual block coordinate ascent for Lagrange decompositions [38]. Despite the demonstrated power of deep networks in representation learning, the relaxation strategy applied to graph matching problem is a weakening of the quadratic assignment problem, which may hurt the performance of graph matching.…”

“…Recently, approaches [17], [18], [19], [20], [21], [22], [23], [24] based on deep neural networks have attracted much research attention due to the ability to learn representative embeddings of nodes and/or edges. It is common for these approaches to embed a differentiable solver for the combinatorial optimization problem into an end-to-end learning framework.…”

“…Different from the well-designed combinatorial solvers in previous learning-free graph matching methods, these learning-based methods usually acquire node correspondences in a more straightforward way by relaxing one or more of the quadratic constraints, discrete constraints and one-to-one matching constraints. Examples include [23], [21], [18] that relax the quadratic assignment problem to a node-wise assignment problem and adopt the Sinkhorn algorithm [25] for the optimization of linear assignment, [17] that works directly on the learned pairwise affinities using a spectral matching algorithm [9] but drops both discrete and one-to-one matching constraints, and [22] that relaxes the graph matching problem based on Lagrangian decomposition and employs dual block coordinate ascent implementations for optimization. Despite remarkable performance gained, these relaxations of the original problem may cause potential limitations on matching performance.…”

Bayesian Deep Graph Matching for Correspondence Identification in Collaborative Perception

“…The identified correspondence can be improved by designing representative graphs [21] or by removing the correspondences violating neighborhood consensus [12]. The accuracy of deep graph matching can be improved by incorporating combinatorial solvers [39], and the efficiency can be improved by decomposing large graphs into small parts [30]. Deep graph matching outperforms traditional learning-free graph matching methods due to its ability to learn from data and its robustness to noise.…”