Narrating by Doing: A Bridging Concept for Understanding and Informing the Design of Tangible Interfaces for Storytelling

Abstract

We present and discuss the concept ‘narrating by doing’ as the process of creating narratives by performing different embodied actions with tangible interfaces for storytelling. We characterize it as a ‘bridging concept’ that can facilitate exchange between theory and design, informing research and design of TUIs for storytelling targeting young children. The concept builds on theories of cognition, learning and narration, specifically drawing upon the following perspectives: Constructionism, Socio-Constructivism, Embodied Cognition, Narratology and The Narrative Practice Hypothesis. Building upon these theoretical foundations, we identify and discuss four ‘design articulations’ (i.e. important parameters that express the qualities of the concept), namely communicative situation, narrative function of the tangible objects, collaborative and embodied actions and the narrator’s position. Finally, we add evidence to the concept and discuss its productiveness by presenting a set of considerations to inform the design of tangible interfaces for storytelling.

Focalization and ideological meanings in a digital multimodal narrative: implications for the construction of the pedagogy of multiliteracies

“…Each app also presents a geographical map that locates the story in the world map; a 360° environment that incentives the children to move their device around to explore and visualize the full environment; an incorporated glossary with keywords from the story and detailed information about the represented culture; a puzzle; and an augmented reality (AR) page that offers children the possibility to print their own AR markers and bring the story protagonists to life in their environment. The kit offers children the possibility to playfully learn about cultural diversity, mediating the digital play: While the story apps inform children about different cultures, they afterwards 'narrate by doing' (Sylla et al 2022), i.e., by using the manipulative and/or the storyMaker: they assemble, record, write, and draw intercultural stories. Several story apps have now been designed by our team of linguists, literacy educators, digital media designers and producers.…”

Meeting the Challenges of Designing Multimodal Texts to Promote Multiliteracies Pedagogy

“…It also includes an augmented reality page allowing children to print their own augmented reality markers and bring the protagonists to life, and an inbuilt glossary with keywords from the story as well as detailed information about the culture represented. The kit enables children to learn about cultural diversity playfully, mediating digital play with "narration by doing" (Sylla et al, 2022) the following activities: reading, assembling, recording, writing, and drawing intercultural stories. Reading with the apps demands certain interactions from the reader (e.g., dragging elements, clicking on objects, moving the device around, etc.)…”