No abstract

We present concurrent theoretical work from HCI and Education that reveals a convergence of trends focused on the importance of three themes: embodiment, multimodality, and composition. We argue that there is great potential for truly transformative work that aligns HCI and Education research, and posit that there is an important opportunity to advance this effort through the full integration of the three themes into a theoretical and technological framework for learning. We present our own work in this regard, introducing the Situated Multimedia Arts Learning Lab (SMALLab). SMALLab is a mixed-reality environment where students collaborate and interact with sonic and visual media through full-body, 3D movements in an open physical space. SMALLab emphasizes human-to-human interaction within a multimodal, computational context. We present a recent case study that documents the development of a new SMALLab learning scenario, a collaborative student participation framework, a student-centered curriculum, and a three-day teaching experiment for seventy-two earth science students. Participating students demonstrated significant learning gains as a result of the treatment. We conclude that our theoretical and technological framework can be broadly applied in the realization of mixed reality, student-centered learning environments.

1. Periodical cicadas exhibit an extraordinary capacity for self-organizing spatially synchronous breeding behavior. The regular emergence of periodical cicada broods across the United States is a phenomenon of longstanding public and scientific interest, as the cicadas of each brood emerge in huge numbers and briefly dominate their ecosystem. During the emergence, the 17-year periodical cicada species Magicicada cassini is found to form synchronized choruses, and we investigated their chorusing behavior from the standpoint of spatial synchrony.2. Cicada choruses were observed to form in trees, calling regularly every five seconds. In order to determine the limits of this self-organizing behavior, we set out to quantify the spatial synchronization between cicada call choruses in different trees, and how and why this varies in space and time.3. We performed 20 simultaneous recordings in Clinton State Park, Kansas, in June 2015 (Brood IV), with a team of citizen-science volunteers using consumer equipment (smartphones). We use a wavelet approach to show in detail how spatially synchronous, self-organized chorusing varies across the forest. 4. We show how conditions that increase the strength of audio interactions between cicadas also increase the spatial synchrony of their chorusing. Higher forest canopy light levels increase cicada activity, corresponding to faster and higheramplitude chorus cycling and to greater synchrony of cycles across space. We implemented a relaxation-oscillator-ensemble model of interacting cicadas, finding that a tendency to call more often, driven by light levels, results in all these effects. 5.Results demonstrate how the capacity to self-organize in ecology depends sensitively on environmental conditions. Spatially correlated modulation of cycling rate by an external driver can also promote self-organization of phase synchrony. K E Y W O R D Scicada, citizen science, insect chorus, synchrony, wavelet

No abstract

This paper presents the design of a multimodal chatbot for use in an interactive theater performance. This chatbot has an architecture consisting of vision and natural language processing capabilities, as well as embodiment in a non-anthropomorphic movable LED array set in a stage. Designed for interaction with up to five users at a time, the system can perform tasks including face detection and emotion classification, tracking of crowd movement through mobile phones, and real-time conversation to guide users through a nonlinear story and interactive games. The final prototype, named ODO, is a tangible embodiment of a distributed multimedia system that solves several technical challenges to provide users with a unique experience through novel interaction.