Cancer affects one in three people worldwide. Surgery remains the primary curative option for localized cancers, but good prognoses require complete removal of primary tumors and timely recognition of metastases. To expand surgical capabilities and enhance patient outcomes, we developed a six-channel color/near-infrared image sensor inspired by the mantis shrimp visual system that enabled near-infrared fluorescence image guidance during surgery. The mantis shrimp’s unique eye, which maximizes the number of photons contributing to and the amount of information contained in each glimpse of its surroundings, is recapitulated in our single-chip imaging system that integrates arrays of vertically stacked silicon photodetectors and pixelated spectral filters. To provide information about tumor location unavailable from a single instrument, we tuned three color channels to permit an intuitive perspective of the surgical procedure and three near-infrared channels to permit multifunctional imaging of optical probes highlighting cancerous tissue. In nude athymic mice bearing human prostate tumors, our image sensor enabled simultaneous detection of two tumor-targeted fluorophores, distinguishing diseased from healthy tissue in an estimated 92% of cases. It also permitted extraction of near-infrared structured illumination enabling the mapping of the three-dimensional topography of tumors and surgical sites to within 1.2-mm error. In the operating room, during surgical resection in 18 patients with breast cancer, our image sensor further enabled sentinel lymph node mapping using clinically approved near-infrared fluorophores. The flexibility and performance afforded by this simple and compact architecture highlights the benefits of biologically inspired sensors in image-guided surgery.

No abstract

No abstract

Dual-mode visible/near-infrared imaging systems, including a bioinspired six-channel design and more conventional four-channel implementations, have transitioned from a niche in surveillance to general use in machine vision. However, the demosaicing routines that transform the raw images from these sensors into processed images that can be consumed by humans or computers rely on assumptions that may not be appropriate when the two portions of the spectrum contribute different information about a scene. A solution can be found in a family of demosaicing routines that utilize interpolating polynomials and splines of different dimensionalities and orders to process images with minimal assumptions.

Real-time guidance through fluorescence imaging improves the surgical outcomes of tumor resections, reducing the chances of leaving positive margins behind. As tumors are heterogeneous, it is imperative to interrogate multiple overexpressed cancer biomarkers with high sensitivity and specificity to improve surgical outcomes. However, for accurate tumor delineation and ratiometric detection of tumor biomarkers, current methods require multiple excitation wavelengths to image multiple biomarkers, which is impractical in a clinical setting. Here, we have developed a biomimetic platform comprising near-infrared fluorescent semiconducting polymer nanoparticles (SPNs) with red blood cell membrane (RBC) coating, capable of targeting two representative cell-surface biomarkers (folate, αυβ3 integrins) using a single excitation wavelength for tumor delineation during surgical interventions. We evaluate our single excitation ratiometric nanoparticles in in vitro tumor cells, ex vivo tumor-mimicking phantoms, and in vivo mouse xenograft tumor models. Favorable biological properties (improved biocompatibility, prolonged blood circulation, reduced liver uptake) are complemented by superior spectral features: (i) specific fluorescence enhancement in tumor regions with high tumor-to-normal tissue (T/NT) ratios in ex vivo samples and (ii) estimation of cell-surface tumor biomarkers with single wavelength excitation providing insights about cancer progression (metastases). Our single excitation, dual output approach has the potential to differentiate between the tumor and healthy regions and simultaneously provide a qualitative indicator of cancer progression, thereby guiding surgeons in the operating room with the resection process.