Developing successful sign language recognition, generation, and translation systems requires expertise in a wide range of fields, including computer vision, computer graphics, natural language processing, human-computer interaction, linguistics, and Deaf culture. Despite the need for deep interdisciplinary knowledge, existing research occurs in separate disciplinary silos, and tackles separate portions of the sign language processing pipeline. This leads to three key questions: 1) What does an interdisciplinary view of the current landscape reveal? 2) What are the biggest challenges facing the field? and 3) What are the calls to action for people working in the field? To help answer these questions, we brought together a diverse group of experts for a two-day workshop. This paper presents the results of that interdisciplinary workshop, providing key background that is often overlooked by computer scientists, a review of the state-of-the-art, a set of pressing challenges, and a call to action for the research community.Each group focused on the following questions:

No abstract

In language, recombination of a discrete set of meaningless building blocks forms an unlimited set of possible utterances. How such combinatorial structure emerged in the evolution of human language is increasingly being studied. It has been shown that it can emerge when languages culturally evolve and adapt to human cognitive biases. How the emergence of combinatorial structure interacts with the existence of holistic iconic form-meaning mappings in a language is still unknown. The experiment presented in this paper studies the role of iconicity and human cognitive learning biases in the emergence of combinatorial structure in artificial whistled languages. Participants learned and reproduced whistled words for novel objects with the use of a slide whistle. Their reproductions were used as input for the next participant, to create transmission chains and simulate cultural transmission. Two conditions were studied: one in which the persistence of iconic form-meaning mappings was possible and one in which this was experimentally made impossible. In both conditions, cultural transmission caused the whistled languages to become more learnable and more structured, but this process was slightly delayed in the first condition. Our findings help to gain insight into when and how words may lose their iconic origins when they become part of an organized linguistic system.

In human speech, a finite set of basic sounds is combined into a (potentially) unlimited set of well-formed morphemes. Hockett (1960) placed this phenomenon under the term ‘duality of patterning’ and included it as one of the basic design features of human language. Of the thirteen basic design features Hockett proposed, duality of patterning is the least studied and it is still unclear how it evolved in language. Recent work shedding light on this is summarized in this paper and experimental data is presented. This data shows that combinatorial structure can emerge in an artificial whistled language through cultural transmission as an adaptation to human cognitive biases and learning. In this work the method of experimental iterated learning (Kirby et al. 2008) is used, in which a participant is trained on the reproductions of the utterances the previous participant learned. Participants learn and recall a system of sounds that are produced with a slide whistle. Transmission from participant to participant causes the whistle systems to change and become more learnable and more structured. These findings follow from qualitative observations, quantitative measures and a follow-up experiment that tests how well participants can learn the emerged whistled languages by generalizing from a few examples.

No abstract