We developed a virtual intraoperative surgical photoacoustic microscopy system by combining with a commercial surgical microscope and photoacoustic microscope (PAM). By sharing the common optical path in the microscope and PAM system, we could acquire the PAM and microscope images simultaneously. Moreover, by employing a beam projector to back-project 2D PAM images onto the microscope view plane as augmented reality, the conventional microscopic and 2D cross-sectional PAM images are concurrently mapped on the plane via an ocular lens of the microscope in real-time. Further, we guided needle insertion into phantom ex vivo and mice skins in vivo. A surgical microscope is a vital tool during operations since it started being used in otolaryngology.1,2 In spite of the continuous improvement of the surgical microscopy performance, it could mainly provide the enlarged surface images without the sub-surface information. Because of this limitation, surgeons are required to have significant experiences and heavy trainings in current clinical practices. Thus, noninvasive visualization of the sub-surface information should play an important role during ophthalmic, microvascular, and neuro-surgeries.
3-5Conventional optical microscopic techniques such as optical coherence tomography (OCT) and fluorescence microscopy (FM) has been applied for this purpose.6-10 OCT mainly provide microstructures of tissues based on optical scattering while the penetration depth of FM is significantly shallow and requires to use the exogenous contrast agents. However, real-time noninvasive mapping of microvasculatures are crucial to verify bleeding regions, guide vascular reconnection, or delineate angiogenesis during the surgeries. 11 To meet this purpose, we are convinced to develop a noninvasive surgical microscopy system to image microvasculatures and surgical intervention without any contrast agent beyond the skin surface in real-time.Photoacoustic microscopy (PAM) is a rising imaging technique that combines optical excitation and ultrasound detection.12-14 Because PAM can provide label-free optical absorption information in a non-invasive manner with high contrast and resolution, this modality has widely been used to image tumor angiogenesis, tumor metabolism, brain function, ocular structures, molecular information, etc. [15][16][17][18][19][20] In this letter, we developed a near-real-time virtual intraoperative surgical photoacoustic microscope (VISPAM) by integrating PAM and conventional surgical microscopy. The VISPAM system could obtain, process, and display both PAM and microscopic images at the same time. The Hilberttransformed cross-sectional PAM B-scan images were projected back onto the microscope view plane as augmented reality through a home-made optical beam splitter, and then the PAM images were shown via the ocular lens mounted on the microscope. Thus, an additional tabletop display is unnecessary, so it is significantly convenient because surgeons potentially do not need to move their sights during the surgeries. We demo...