Development of an App and Teaching Concept for Implementation of Hyperspectral Remote Sensing Data into School Lessons Using Augmented Reality

Abstract

For the purpose of expanding STEM (science, technology, engineering, mathematics) education with remote sensing (RS) data and methods, an augmented reality (AR) app was developed in combination with a worksheet and lesson plan. Data from the Hyperspectral Imager for the Coastal Ocean (HICO) was searched for topics applicable to STEM curricula, which was found in the example of a harmful algal bloom in Lake Erie, USA, in 2011. Spectral shape algorithms were applied to differentiate between less harmful green and more harmful blue algae in the lake. The data was pre-processed to reduce its size significantly without losing too much information and then integrated into an app that was developed in Unity with the Vuforia extension. It was designed to let students browse and understand the raw data in RGB and a tangible hyperspectral cube, as well as to analyze algae maps derived from it. The app runs on Android smartphones with minimized data usage to make it less dependent on school funding and the socioeconomic background of students. Using educational concepts, such as active and collaborative learning, moderate constructivism, and scientific inquiry, the data was integrated into a lesson about environmental problems that was enhanced by the AR app. The app and worksheet were evaluated in two advanced geography courses (n = 36) and found to be complex, but doable and understandable, for the target group of German high school students in their final two school years. Thus, hyperspectral data can be used for STEM lessons using AR technology on students’ smartphones with several limitations both in the technology used and gainable knowledge.

Integrating immersive technologies with STEM education: a systematic review

“…• Kao and Ruan (2022), Lindner et al (2022), andCampos et al (2022) found increased engagement and performance through interactive AR modes and applications in programming and environmental science.…”

SatelliteSkill5—An Augmented Reality Educational Experience Teaching Remote Sensing through the UN Sustainable Development Goals

“…Unity is a powerful, versatile, and popular game development engine that provides a sandbox environment for the creation of immersive and interactive 2D and 3D experiences. Unity (versions 2019-2022.3 used during development) offers exceptional crossplatform capabilities, including mobile devices, consoles, and desktops, and it supports emerging technologies such as AR (e.g., [30][31][32][33]) and VR, which were used by the Satel-liteSkill5 app; many of these AR examples were carried out with public engagement [34,35] or education and capacity building [36,37] in mind. The user interface that Unity provides significantly simplifies the game development process without the developer needing extensive coding knowledge.…”

Developing and Evaluating Simplified Tools for Image Processing in a Problem-Based Learning Environment for Earth Observation

“…It is, however, solely focused on geography lessons. Another recent approach to integrating remote sensing into classrooms makes use of the BYOD (Bring Your Own Device) concept, integrating the capabilities of pupil's smartphones in learning experiences based on augmented reality (Lindner et al 2019(Lindner et al , 2022López Núñez et al 2020;Rienow et al 2020). While they show promising capabilities and offer interesting approaches to even include topics like hyperspectral imaging in classrooms (Lindner et al 2022) they are limited in terms of computational capacities of the devices used.…”