Integration of 3D Gaussian Splatting and Neural Radiance Fields in Virtual Reality Fire Fighting

Abstract

Neural radiance fields (NeRFs) and 3D Gaussian splatting have emerged as promising 3D reconstruction techniques recently. However, their application in virtual reality (VR), particularly in firefighting training, remains underexplored. We present an innovative VR firefighting simulation system based on 3D Gaussian Splatting technology. Leveraging these techniques, we successfully reconstruct realistic physical environments. By integrating the Unity3D game engine with head-mounted displays (HMDs), we created and presented immersive virtual fire scenes. Our system incorporates NeRF technology to generate highly realistic models of firefighting equipment. Users can freely navigate and interact with fire within the virtual fire scenarios, enhancing immersion and engagement. Moreover, by utilizing the Photon PUN2 networking framework, our system enables multi-user collaboration on firefighting tasks, improving training effectiveness and fostering teamwork and communication skills. Through experiments and surveys, it is demonstrated that the proposed VR framework enhances user experience and holds promises for improving the effectiveness of firefighting training.