Digital Twin Assessments in Virtual Reality: An Explorational Study with Aeroengines

Abstract

We present an immersive environment where Virtual Reality (VR) is used to visualize the performance of a fleet of aircraft engines. Our virtual environment uses 3D geometric computer-aided design (CAD) models of the engines paired with performance maps that characterize their nominal working condition. These maps plot pressure ratio and efficiency as a function of shaft speed and inlet flow capacity for the numerous engine sub-systems. Superimposed on these maps is the true performance of each engine, obtained through real-time sensors. In this bespoke virtual space, an engineer can rapidly analyze the health of different engine sub-systems across the fleet within seconds. One of the key elements of such a system is the selection of an appropriate interaction technique. In this paper we explore the potential of interaction methods supported by a combination of gaze-tracking and hand-tracking achieved via an additional sensor attached to the front of the VR headset, with no need for the user to hold a controller. We report on an observational study with a small number of domain-experts to identify usability problems, spot potential improvements, and gain insights into our design interaction capabilities. The study allows us to trim the design space and to guide further design efforts in this area. We also analyze qualitative feedback provided by the end-users and discuss the lessons learned during the design, implementation, verification and validation of the system.

Measurement and Inspection of Photo-Realistic 3-D VR Models

“…We used established survey instruments to assess simulator sickness (SSQ) [6], flow short scale (FSS) [7], sense of presence (IPQ) 4 , and perceived cognitive load (NASA TLX) [8]. For SSQ, P1 reported no symptoms while P2 and P3 reported mild symptoms.…”

“…The hardware setup consisted of the Oculus Rift HMD, its position-tracking sensor, and the Leap Motion hand-tracking sensor. The basis of the VR framework underpinning the system was previously developed for aeroengine digital twin assessment [4] and used in another qualitative user study [5]. This prior user study [5] was concerned with a different photogrammetric model and the participants in that study were chosen due to their expertise in civil engineering rather than piping systems.…”

Exploring gestural input for engineering surveys of real-life structures in virtual reality using photogrammetric 3D models

“…Such information can be used by experts to reason about the physical state of a given structure. This is not possible when working with computer-aided design (CAD) models, which we have investigated in prior studies [26]. Even carefully crafted CAD models do not represent the current visible state of real-life structures, as they are not generated based on recently captured data.…”

Digital Twin as Industrial Robots Manipulation Validation Tool

“…Direct manipulation proves to be the best type of interface for the given task overall. A VR DT interface for aircraft engines performance control is presented by Tadeja et al [9]. Information and nominal performance maps are synchronized with both real and digital representations of engines, allowing for real-time visualization and manipulation in the immersive environment.…”