Sequence-to-sequence models are a powerful workhorse of NLP. Most variants employ a softmax transformation in both their attention mechanism and output layer, leading to dense alignments and strictly positive output probabilities. This density is wasteful, making models less interpretable and assigning probability mass to many implausible outputs. In this paper, we propose sparse sequence-to-sequence models, rooted in a new family of α-entmax transformations, which includes softmax and sparsemax as particular cases, and is sparse for any α > 1. We provide fast algorithms to evaluate these transformations and their gradients, which scale well for large vocabulary sizes. Our models are able to produce sparse alignments and to assign nonzero probability to a short list of plausible outputs, sometimes rendering beam search exact. Experiments on morphological inflection and machine translation reveal consistent gains over dense models. arXiv:1905.05702v2 [cs.CL] 12 Jun 2019 the encodings [h 1 , . . . , h J ], using s t as a query vector. This is done by computing token-level scores z j := s t W (z) h j , then taking a weighted averageπ j h j , where π := softmax(z).(1) the anonymous reviewers, for helpful discussion and feedback.

Grapheme-to-phoneme conversion (g2p) is necessary for text-to-speech and automatic speech recognition systems. Most g2p systems are monolingual: they require language-specific data or handcrafting of rules. Such systems are difficult to extend to low resource languages, for which data and handcrafted rules are not available. As an alternative, we present a neural sequence-to-sequence approach to g2p which is trained on spelling-pronunciation pairs in hundreds of languages. The system shares a single encoder and decoder across all languages, allowing it to utilize the intrinsic similarities between different writing systems. We show an 11% improvement in phoneme error rate over an approach based on adapting high-resource monolingual g2p models to low-resource languages. Our model is also much more compact relative to previous approaches.

No abstract

This paper presents the Instituto de Telecomunicações-Instituto Superior Técnico submission to Task 1 of the SIGMORPHON 2019 Shared Task. Our models combine sparse sequence-to-sequence models with a two-headed attention mechanism that learns separate attention distributions for the lemma and inflectional tags. Among submissions to Task 1, our models rank second and third. Despite the low data setting of the task (only 100 in-language training examples), they learn plausible inflection patterns and often concentrate all probability mass into a small set of hypotheses, making beam search exact.

The Massachusetts Institute of Technology (MIT) Mediated Matter Group is honing its research into robotic swarm printing by focusing its efforts on material sophistication, or ‘tunability’, and communication or coordination between fabrication units. Here, the group's Neri Oxman, Jorge Duro‐Royo, Steven Keating, Ben Peters and Elizabeth Tsai illustrate this by describing three case studies that investigate robotically controlled additive fabrication at architectural scales.