Summary: The technology used to add information to a real visual field is defined as augmented reality technology. Augmented reality technology that can interactively manipulate displayed information is called mixed reality technology. HoloLens from Microsoft, which is a head-mounted mixed reality device released in 2016, can display a precise three-dimensional model stably on the real visual field as hologram. If it is possible to accurately superimpose the position/direction of the hologram in the surgical field, surgical navigation-like use can be expected. However, in HoloLens, there was no such function. The authors devised a method that can align the surgical field and holograms precisely within a short time using a simple manual operation. The mechanism is to match the three points on the hologram to the corresponding marking points of the body surface. By making it possible to arbitrarily select any of the three points as a pivot/axis of the rotational movement of the hologram, alignment by manual operation becomes very easy. The alignment between the surgical field and the hologram was good and thus contributed to intraoperative objective judgment. By using the method of this study, the clinical usefulness of the mixed reality device HoloLens will be expanded.

Background: Telementoring is the technology for providing surgical instruction from a remote place via a network. To demonstrate the use of telementoring in craniofacial surgery, Skype and a mixed reality device HoloLens were adopted, and 3-layer facial models had been developed. Methods: A resident in hospital A used the model surgery under remote guidance by a mentor surgeon in hospital B 4 times on different dates. The straight-line between hospitals A and B is 250 km. The mentor gave the resident guidance via Skype and HoloLens, communicating by voice, and video of the surgical field, and providing reference data. Results: There was no delay in voice communication and a delay of <0.5 seconds in the video. The resident was able to confirm the main landmarks of the surgical field and to grasp the situation without problems. The mentor could send appropriate instructions by voice, could point out a specific part by telestration function, and could draw lines on the 2-dimentional images pasted on the operator's field of vision. Discussion: With the use of HoloLens, Skype, and the 3-layer models, it was possible to demonstrate telementoring. The risk of personal information leakage due to data interception seems to be very low because its data communication is encrypted with advanced encryption standard. Conclusion: This telementoring system has various advantages and many improvable aspects in the field of craniofacial surgery.

Background:Augmented reality (AR) technology that can combine computer-generated images with a real scene has been reported in the medical field recently. We devised the AR system for evaluation of improvements of the body surface, which is important for plastic surgery.Methods:We constructed an AR system that is easy to modify by combining existing devices and free software. We superimposed the 3-dimensional images of the body surface and the bone (obtained from VECTRA H1 and CT) onto the actual surgical field by Moverio BT-200 smart glasses and evaluated improvements of the body surface in 8 cases.Results:In all cases, the 3D image was successfully projected on the surgical field. Improvement of the display method of the 3D image made it easier to distinguish the different shapes in the 3D image and surgical field, making comparison easier. In a patient with fibrous dysplasia, the symmetrized body surface image was useful for confirming improvement of the real body surface. In a patient with complex facial fracture, the simulated bone image was useful as a reference for reduction. In a patient with an osteoma of the forehead, simultaneously displayed images of the body surface and the bone made it easier to understand these positional relationships.Conclusions:This study confirmed that AR technology is helpful for evaluation of the body surface in several clinical applications. Our findings are not only useful for body surface evaluation but also for effective utilization of AR technology in the field of plastic surgery.

To make three-dimensional computer-assisted elastic models for the face, we decided on five requirements: (1) an elastic texture like skin and subcutaneous tissue; (2) the ability to take pen marking for incisions; (3) the ability to be cut with a surgical knife; (4) the ability to keep stitches in place for a long time; and (5) a layered structure. After testing many elastic solvents, we have made realistic three-dimensional computer-assisted two-layer elastic models of the face and cleft lip from the computed tomographic and magnetic resonance imaging stereolithographic data. The surface layer is made of polyurethane and the inner layer is silicone. Using this elastic model, we taught residents and young doctors how to make several typical local flaps and to perform cheiloplasty. They could experience realistic simulated surgery and understand three-dimensional movement of the flaps.

Background: The positioning of the auricle is a key factor in successful ear reconstruction. However, the position of the ear is usually determined by transferring the auricle image of the nonaffected side to the affected side using a transparent film. Augmented reality (AR) is becoming useful in the surgical field allowing computer-generated images to be superimposed on patients. In this report, we would like to introduce an application of AR technology in ear reconstruction. Methods: AR technology was used to determine the position of the reconstructed ear of a 10-year-old male with right microtia. Preoperative 3-dimensional photographs of the nonaffected side were taken using VECTRAH1. Then, the image was horizontally inverted and superimposed on the three-dimensional image of the affected side with reference to the anatomical landmarks of the patient’s face. These images were projected onto the patient in the operation room using Microsoft’s HoloLens. The design and positioning of the auricle was done with reference to the AR image. To confirm the accuracy of the AR technique, we compared it to the original transparent film technique. After the insertion of the cartilage framework into the skin pocket, the position and shape of the reconstructed ear was confirmed using the AR technology. Results: The positioning of the reconstructed ear was successfully performed. The deviation between the 2 designated positions using the AR and the transparent film was within 2 mm. Conclusion: The AR technology is a promising option in the surgical treatment of microtia.