Deep Siamese Metric Learning: A Highly Scalable Approach to Searching Unordered Sets of Trajectories

Abstract

This work proposes metric learning for fast similarity-based scene retrieval of unstructured ensembles of trajectory data from large databases. We present a novel representation learning approach using Siamese Metric Learning that approximates a distance preserving low-dimensional representation and that learns to estimate reasonable solutions to the assignment problem. To this end, we employ a Temporal Convolutional Network architecture that we extend with a gating mechanism to enable learning from sparse data, leading to solutions to the assignment problem exhibiting varying degrees of sparsity. Our experimental results on professional soccer tracking data provides insights on learned features and embeddings, as well as on generalization, sensitivity, and network architectural considerations. Our low approximation errors for learned representations and the interactive performance with retrieval times several magnitudes smaller shows that we outperform previous state of the art.

Active Learning of Ordinal Embeddings: A User Study on Football Data

“…Tuple composition and sample selection are important for learning an ordinal embedding specific to a human's similarity function. We formulate the problem first as pairwise similarity learning using a Siamese network like Löffler et al (2021), that initially approximates assignments through metric learning (Section 2.1). This delivers a meaningful embedding to warm-start tuple selection strategies.…”

“…In large and complex datasets of unstructured data, such as in multi-agent positional tracking in sports (Löffler et al, 2021;Sha et al, 2016) or transportation (Yadamjav et al, 2020), information retrieval requires two expensive steps. First, the unstructured trajectories are optimally assigned, e.g., using the Hungarian algorithm Kuhn (1955).…”

“…First, the Euclidean distance in itself is limiting, e.g., it does not weigh subjectively more important trajectories higher, and is affected by the data's high dimensionality (Canessa et al, 2020). Second, following directly from estimating the Euclidean distance of high dimensional data, the embedding captures global ordinal structures better than local ones (Löffler et al, 2021). This work proposes to actively learn a distance function from human annotations.…”

“…Specifically, we shed light on the strengths and weaknesses of passive sampling heuristics and active learners alike by analyzing the participants' response efficacy. To this end, we collect accuracy, algorithmic time complexity, the participants' fatigue and time-to-response, qualitative self-assessment and statements, as well as the effects of mixed-expertise annotators and their consistency on model performance and transfer-learning.Recently, convolutional Siamese networks were leveraged by Löffler et al (2021) to learn approximations of both the trajectory assignment and distance metric. The resulting lower-dimensional representation enables 1…”

“…Recently, convolutional Siamese networks were leveraged by Löffler et al (2021) to learn approximations of both the trajectory assignment and distance metric. The resulting lower-dimensional representation enables 1…”