Exploitation of multiplayer interaction and development of virtual puppetry storytelling using gesture control and stereoscopic devices

Liang, Hui; Chang, Jian; Deng, Shujie; Chen, Can; Tong, Ruofeng; Zhang, Jian Jun

doi:10.1002/cav.1727

Cited by 16 publications

(8 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An increase in interactivity from a software perspective would be accomplished by programming the virtual object to respond appropriately to multiple forms of user input or by increasing user-friendliness. Purposes of interactivity may have included virtual environments that were designed to feature optimal accessibility, such as the virtual multiplayer child-operated puppet story as presented by Liang et al (2017).…”

Section: Interactivitymentioning

confidence: 99%

“…A user with a university background in mechanical engineering would most likely have no trouble utilizing complex button-operated input controllers to interact with a virtual object (e.g., Virtual Workshop as reported by Muller et al (2017). Likewise, a user who is inexperienced with technical hardware would likely benefit with a simpler input device to interact with virtual objects [e.g., Liang et al (2017) child-operated virtual puppet story with gesture control, detected by Leap Motion].…”

Section: Considerations Of Virtual System Designmentioning

confidence: 99%

See 1 more Smart Citation

Head-Mounted Display Virtual Reality in Post-secondary Education and Skill Training

2019

View full text Add to dashboard Cite

Background: This review focused on how immersive head-mounted display virtual reality (VR) was used in post-secondary level education and skill training, with the aim to better understand its state of the art as found from the literature. While numerous studies describe the use of immersive VR within a specific educational setting, they are often standalone events not fully detailed regarding their curricular integration. This review aims to analyse these events, with a focus on immersive VR's incorporation into post-secondary education.Objectives: (O1) Review the existing literature on the use of immersive VR in post-secondary settings, determining where and how it has been used within each educational discipline. This criterion focused on literature featuring the use of immersive VR, due to its influence on a user's perceived levels of presence and imagination. (O2) Identify favorable outcomes from the use of immersive VR when it is compared to other learning methods. (O3) Determine the conceptual rationale (purpose) for each implementation of immersive VR as found throughout the literature. (O4) Identify learning theories and recommendations for the utilization of immersive VR in post-secondary education.Methods: A literature review was undertaken with searches of Education Research Complete, ERIC, MEDLINE, EMBASE, IEEE Xplore, Scopus, and Web of Science: Core Collection to locate reports on the use of immersive VR in post-secondary curricula.Results: One hundred and nineteen articles were identified, featuring disciplines across Arts and Humanities, Health Sciences, Military and Aerospace, Science and Technology. Thirty five out of 38 experiments reported to have found a positive outcome for immersive VR, after being compared with a non-immersive platform. Each simulation's purpose included one or more of the following designations: skill training, convenience, engagement, safety, highlighting, interactivity, team building, and suggestion. Recommendations for immersive VR in post-secondary education emphasize experiential learning and social constructivist approaches, including student-created virtual environments that are mainly led by the students themselves under team collaboration.Concannon et al. Immersive VR in Post-secondary EducationConclusion: Immersive VR brings convenient, engaging, and interactive alternatives to traditional classroom settings as well as offers additional capability over traditional methods. There is a diverse assortment of educational disciplines that have each attempted to harness the power of this technological medium.

show abstract

Section: Interactivitymentioning

confidence: 99%

Section: Considerations Of Virtual System Designmentioning

confidence: 99%

Head-Mounted Display Virtual Reality in Post-secondary Education and Skill Training

2019

View full text Add to dashboard Cite

show abstract

“…Storytelling has a rich tradition and is considered as one of the original teaching forms (Liang et al 2017). Storytelling is central in puppetry, and De Beer, Petersen and Brits (2018) explored the epistemological border-crossing, using puppets to infuse art and drama into the classroom.…”

Section: Puppetry As a Teaching Methodologymentioning

confidence: 99%

Arguing for the inclusion of indigenous knowledge in the STEM curriculum: Possibilities and challenges

Beer¹,

Wyk²

2019

NWU Self-Directed Learning Series

View full text Add to dashboard Cite

Peer review declarationThe publisher (AOSIS) endorses the South African 'National Scholarly Book Publishers Forum Best Practice for Peer Review of Scholarly Books'. The manuscript was subjected to a rigorous two-step peer review prior to publication, with the identities of the reviewers not revealed to the author(s). The reviewers were independent of the publisher and/or authors in question. The reviewers commented positively on the scholarly merits of the manuscript and recommended that the manuscript should be published. Where the reviewers recommended revision and/or improvements to the manuscript, the authors responded adequately to such recommendations. Research JustificationThis book is the result of a longitudinal research project (2016)(2017)(2018) funded by the National Research Foundation and the Fuchs Foundation, and it disseminates original research. The project researched the affordances of indigenous knowledge in the school Science, Technology and Mathematics curricula. Short learning programmes (SLPs) were offered to Science, Technology, Engineering and Mathematics (STEM) teachers, during which they engaged in creative and inquiry-based teaching and learning strategies. Research shows that strategies such as problem-based and cooperative learning have the potential to enhance self-directed learning. This design-based research was conducted in several provinces in South Africa (North-West Province, the Northern Cape, Limpopo province, and Gauteng). Based on the data obtained after each intervention, design principles were formulated for redesigning of SLPs. The qualitative research focussed on teachers' lived experiences of the epistemological bordercrossing between Natural Science and indigenous knowledge, their views on the nature of science and indigenous knowledge, and the reformed teaching and learning that took place after the intervention in teachers' classrooms. Most of the chapters in the book report empirical data, with the exception of three chapters (Ch. 2, Ch. 10 and Ch. 11), that can be categorised as systematic reviews. The book is devoted to scholarship in the field of STEM education and teacher professional development, with a specific focus on research into the enhancement of self-directed learning. The target audience are scholars working in the fields of indigenous knowledge systems, STEM education, teacher professional development and self-directed learning. This book makes a unique contribution in terms of, firstly, its extensive use of third-generation Cultural-Historical Activity Theory (CHAT) as a research lens and, secondly, in drawing on research from the fields of neuroscience, science education philosophy, self-directed learning and indigenous knowledge systems, in arguing for such border-crossing. Chapter 1 shows how this research contributes to the development of fourthgeneration CHAT, which, owing to the complexity of the object (which Engeström refers to as 'runaway objects'), requires a focus on numerous activity systems. The book also explores the conundrum of rese...

show abstract

“…In addition to research related to virtual (avatar) models in traditional culture, there are other studies and applications regarding virtual model control. For example, in a research study [14] similar to Taiwanese traditional puppetry, a gesture-based multi-user digital storytelling system was developed using VR technology and Leap Motion device. This system is primarily designed to assist children in storytelling and performance.…”

Section: Related Workmentioning

confidence: 99%

An IoT-Based Microservice Platform for Virtual Puppetry Performance

Luo,

Lin,

Liao

et al. 2023

IEEE Access

View full text Add to dashboard Cite

This study proposes a web-based real-time remote-controlled virtual puppetry (avatar) performance platform called AvatarTalk to combine traditional Taiwanese puppetry, the Internet of Things (IoT), and 3D avatar models. AvatarTalk enables puppeteers to control avatar puppets on the web using either motion capture gloves or camera-based image recognition. This transformative performance approach not only facilitates remote performances and multi-screen presentations across various devices but also introduces a fresh gesture interpretation technique for dance artists. AvatarTalk supports a mechanism that can accommodate new control and puppet devices from other approaches through the IoT-based microservice concept. We develop a calibration procedure to enable the accurate capture of hand gestures to manipulate the movements of virtual puppets, empowering them to perform fundamental traditional puppetry poses such as nodding, bowing, and synchronized hand movements. We have conducted experiments to show the accuracy of AvatarTalk calibration. Our study indicates that AvatarTalk can almost detect the right gestures (98.75%-100% recall) and very seldom mistake the wrong gestures (90.8%-100% precision). Additionally, we also provide the mechanism to measure the delays of controlling the puppets. An analytic model is proposed to design the delay times of the messages between the control device of a puppeteer and the AvatarTalk server. In the current AvatarTalk implementation, if the message delay does not exceed 0.1 seconds, four puppeteers can synchronize their actions if the elapsed time between two actions of a puppeteer is longer than 0.3 second.

show abstract

Exploitation of multiplayer interaction and development of virtual puppetry storytelling using gesture control and stereoscopic devices

Cited by 16 publications

References 33 publications

Head-Mounted Display Virtual Reality in Post-secondary Education and Skill Training

Head-Mounted Display Virtual Reality in Post-secondary Education and Skill Training

Arguing for the inclusion of indigenous knowledge in the STEM curriculum: Possibilities and challenges

An IoT-Based Microservice Platform for Virtual Puppetry Performance

Contact Info

Product

Resources

About