Flying a helicopter with the HoloLens as head-mounted display

Abstract

We describe the flight testing and the integration process of the Microsoft HoloLens 2 as head-mounted display (HMD) with DLR's research helicopter. In the previous work, the HoloLens was integrated into a helicopter simulator. Now, while migrating the HoloLens into a real helicopter, the main challenge was the head tracking of the HoloLens, because it is not designed to operate on moving vehicles. Therefore, the internal head tracking is operated in a limited rotation-only mode, and resulting drift errors are compensated for with an external tracker, several of which have been tested in advance. The fusion is done with a Kalman filter, which contains a non-linear weighting. Internal tracking errors of the HoloLens caused by vehicle accelerations are mitigated with a system identification approach. For calibration, the virtual world is manually aligned using the helicopter's noseboom. The external head tracker (EHT) is largely automatically calibrated using an optimization approach and therefore, works for all trackers and regardless of its mounting positions on vehicle and head. Most of the pretests were carried out in a car, which indicates the flexibility in terms of vehicle type. The flight tests have shown that the overall quality of this HMD solution is very good. The conformal holograms are almost jitter-free, there is no latency, and errors of lower frequencies are identical with the performance that the EHT can provide, which in combination greatly improves immersion. Profiting from almost all features of the HoloLens 2 is a major advantage, especially for rapid research and development. © The Authors. Published by SPIE under a Creative Commons Attribution 4.0 International License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.

High‐Performance Transparent Silicon Nanowire Thin Film Transistors Integrated on Glass Substrates via a Room Temperature Solution Passivation

“…For the SiNW TFT samples with ITO electrodes on glass substrate, the whole samples look even more transparent (see Figure 2i), and even for the sample with opaque metal electrodes, the overall transparency is ≈73% (as witnessed in Figure S2, Supporting Information), which is sufficient for most transparent electronics and AR/VR display applications. [46] The typical transfer and output characteristics of the SiNW-TFTs fabricated on the wafer substrates, via an all-low-temperature (<350 °C) growth and solution passivation process (for two cycles) at RT, are first presented in Figure 3a,b, where the p-type transfer property arises from the incorporation of In atoms into the c-SiNW channels during the IPSLS growth. [42,47] Under V ds = 0.5 V bias, a high On/Off current ratio of ≈4 × 10 6 and a steep SS of 120 mV dec −1 have been achieved.…”

“…For the SiNW TFT samples with ITO electrodes on glass substrate, the whole samples look even more transparent (see Figure 2i), and even for the sample with opaque metal electrodes, the overall transparency is ≈73% (as witnessed in Figure S2, Supporting Information), which is sufficient for most transparent electronics and AR/VR display applications. [ 46 ]…”

Microsoft HoloLens 2 in Medical and Healthcare Context: State of the Art and Future Prospects

“…The full texts of 83 papers assessed for eligibility were carefully analyzed. Thirteen articles [ 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 ] were excluded due to the exclusion criteria (1), one contribution [ 29 ] due to the exclusion criteria (2), 17 scientific results [ 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 ] due to the exclusion criteria (3) and 5 [ 47 , 48 , 49 , 50 , 51 ] contributions due to the exclusion criteria (4). Finally, only 47 studies were included in the quantitative synthesis.…”

Visual augmentation of deck-landing-ability improves helicopter ship landing decisions

“…The key factors underlying pilots’ safety and performances have therefore been extensively studied 1 , 2 , 3 , 4 , 5 . Several research teams (such as Tu-Deft 6 , University of Liverpool 7 , University of Southampton 8 , and DLR, the German Aerospace Center 9 ) have been investigating the tremendous possibilities offered by Augmented Reality (AR) methods in order to further improve pilots’ safety and their performances. However, visual Augmentations must be carefully designed to optimize their beneficial effects on human performances while preventing the occurrence of detrimental effects.…”