Set-to-Sequence Methods in Machine Learning: a Review

Abstract

Machine learning on sets towards sequential output is an important and ubiquitous task, with applications ranging from language modelling and meta-learning to multi-agent strategy games and power grid optimization. Combining elements of representation learning and structured prediction, its two primary challenges include obtaining a meaningful, permutation invariant set representation and subsequently utilizing this representation to output a complex target permutation. This paper provides a comprehensive introduction to the field as well as an overview of important machine learning methods tackling both of these key challenges, with a detailed qualitative comparison of selected model architectures.

Learning Prototype-oriented Set Representations for Meta-Learning

“…Machine learning models, such as convolutional neural networks for images [13] and recurrent neural networks for sequential data [36], have achieved great success in taking advantage of the structure in the input space [27]. However, extending them to handle unstructured input in the form of sets, where a set can be defined as an unordered collections of elements, is not trivial and has recently attracted increasing attention [20]. Set-input is relevant to a range of problems, such as understanding a scene formed of a set of objects [10], classifying an object composed of a set of 3D points [32], and estimating summary statistics from a set of data points for implicit generative models [5].…”