RescueAR: Augmented Reality Supported Collaboration for UAV Driven Emergency Response Systems

Abstract

Fig. 1. RescueAR utilizes location-based augmented reality technology to facilitate the exchange of information among rescue team members during a UAV-driven emergency response. (a) Flying-AR : Allows remote monitoring teams to mark the injured person in the aerial video stream from a UAV. (b) FirstResponse-AR : Allows on-scene first response team members to visualize the location of the identified injured victim in (a) using AR glasses or mobile devices. (c) Mission-Control : Allows mission planning team to visualize the spatial summary of the mission including the location of UAVs, their aerial video stream, and points of interest identified via (a) Emergency response events are fast-paced, noisy, and they require teamwork to accomplish the mission. Furthermore, the increasing deployment of Unmanned Aerial Vehicles (UAVs) alongside emergency responders, demands a new form of partnership between humans and UAVs. Traditional radio-based information exchange between humans during an emergency response suffers from a lack of visualization and often results in miscommunication. This paper presents a novel collaboration platform: RescueAR, which utilizes the paradigm of Location-based Augmented Reality to geotag, share, and visualize information. RescueAR aims to support the two-way communication between humans and UAVs, facilitate collaboration across diverse responders, and visualize scene information relevant to the rescue team's role. According to our feasibility study, user study, followed by a focus group session with police officers, RescueAR can support rescue teams in developing the spatial cognition of the scene, facilitate the exchange of geolocation information, and complement existing communication tools during the UAV-supported emergency response. CCS Concepts: • Human-centered computing → Collaborative and social computing.

Exploring the Design Space of Optical See-through AR Head-Mounted Displays to Support First Responders in the Field

“…As a result, we collected a final list of 27 papers that focused on AR technologies for FRs, and identified four types of AR cues, including: 1) maps for space visualization and overview [22,24,35,37,50,66,70,80,90,92]; 2) icons and text to label point of interests and provide detailed information [5,22,35,37,38,50,59,67,70,79,90,92]; 3) arrows and paths for navigation [36,37,66,80]; and 4) outline and shaded overlay for object highlighting and enhancement [14,37].…”

“…2D) [14]; and (4) labeling cues that labels objects or human of interest with icons and text (Fig. 2G-H) [5,22,35,37,67,92]. We implemented the AR cues on HoloLens 2 using the Mixed Reality Feature Tool [63] in Unity 2020.3.33.…”

“…(4) Are you currently a first responder or have been one in the past? (5) Please specify your years of experiences as a first responder with start and end years (e.g., 2010 -2015; 2020 -present; 3 months). (6) Do you have any prior experiences with AR glasses?…”

“…(4) What is your specific occupation and title? (5) What is your duty? (6) What are the common tasks you need to perform to fulfill your duty?…”

UAV control in autonomous object-goal navigation: a systematic literature review

“…This SLR seeks to identify the state-of-the-art object-goal navigation methods using UAVs for an emergency response scenario (Agrawal and Cleland-Huang 2021). Two main UAV types were found.…”