Hexachromatic bioinspired camera for image-guided cancer surgery

Abstract

Cancer affects one in three people worldwide. Surgery remains the primary curative option for localized cancers, but good prognoses require complete removal of primary tumors and timely recognition of metastases. To expand surgical capabilities and enhance patient outcomes, we developed a six-channel color/near-infrared image sensor inspired by the mantis shrimp visual system that enabled near-infrared fluorescence image guidance during surgery. The mantis shrimp’s unique eye, which maximizes the number of photons contributing to and the amount of information contained in each glimpse of its surroundings, is recapitulated in our single-chip imaging system that integrates arrays of vertically stacked silicon photodetectors and pixelated spectral filters. To provide information about tumor location unavailable from a single instrument, we tuned three color channels to permit an intuitive perspective of the surgical procedure and three near-infrared channels to permit multifunctional imaging of optical probes highlighting cancerous tissue. In nude athymic mice bearing human prostate tumors, our image sensor enabled simultaneous detection of two tumor-targeted fluorophores, distinguishing diseased from healthy tissue in an estimated 92% of cases. It also permitted extraction of near-infrared structured illumination enabling the mapping of the three-dimensional topography of tumors and surgical sites to within 1.2-mm error. In the operating room, during surgical resection in 18 patients with breast cancer, our image sensor further enabled sentinel lymph node mapping using clinically approved near-infrared fluorophores. The flexibility and performance afforded by this simple and compact architecture highlights the benefits of biologically inspired sensors in image-guided surgery.

Intraoperative fluorescence molecular imaging accelerates the coming of precision surgery in China

“…Although the cancer specificity of these imaging agents is limited, the FMI performance could be improved by adjusting the agent administration. Moreover, combined employment of these fluorescent agents could be an effective and practical way to further amend the surgical outcome [ 56 ].…”

“…However, the goal to specifically identify the metastatic nodes remains unachieved using ICG. Employing newly designed molecular-targeted imaging agents might overcome this problem [ 78 ], and multi-spectrum imaging is also believed as a feasible approach [ 56 ].…”

Photodetectors Based on Lead Sulfide Quantum Dot and Organic Absorbers for Multispectral Sensing in the Visible to Short‐Wave Infrared Range

“…Since the signals coming from the two sources are highly different in intensity, the readout electronics could be configured such that both signals can be detected optimally. [ 24 ]…”

Protease-activated indocyanine green nanoprobes for intraoperative NIR fluorescence imaging of primary tumors

“…6 Various imaging techniques, such as optical coherence tomography, photoacoustic imaging, label-free multispectral imaging, phase-contrast imaging, and uorescence imaging are actively investigated and evaluated in clinical settings for identifying and differentiating cancerous from healthy tissue during intraoperative procedures. [7][8][9][10][11][12][13] Among these imaging techniques, near-infrared (NIR) uorescence imaging has shown great potential to facilitate cancer surgeries. Imaging in the NIR spectrum (700-1000 nm) has the inherent benets of deep tissue imaging due to minimal light absorption by the primary constituents of tissue, i.e., water and hemoglobin, high tumor-to-background ratio (TBR) due to low tissue auto-uorescence, and high patient and caregiver safety due to the use of non-radiation energy.…”