An Insect Eye Inspired Miniaturized Multi-Camera System for Endoscopic Imaging

Abstract

In this work, we present a miniaturized high definition vision system inspired by insect eyes, with a distributed illumination method, which can work in dark environments for proximity imaging applications such as endoscopy. Our approach is based on modeling biological systems with off-the-shelf miniaturized cameras combined with digital circuit design for real time image processing. We built a 5 mm radius hemispherical compound eye, imaging a 180×180 degrees field of view while providing more than 1.1 megapixels (emulated ommatidias) as real-time video with an inter-ommatidial angle ∆ϕ = 0.5 at 18 mm radial distance. We made an FPGA implementation of the image processing system which is capable of generating 25 fps video with 1080 × 1080 pixel resolution at a 120 MHz processing clock frequency. When compared to similar size insect eye mimicking systems in literature, the system proposed in this paper features 1000 × resolution increase. To the best of our knowledge, this is the first time that a compound eye with built-in illumination idea is reported. We are offering our miniaturized imaging system for endoscopic applications like colonoscopy or laparoscopic surgery where there is a need for large field of view high definition imagery. For that purpose we tested our system inside a human colon model. We also present the resulting images and videos from the human colon model in this paper.

Mining the Smartness of Insect Ultrastructures for Advanced Imaging and Illumination

“…Compound eye structures in insects, combined with ultrathin imaging systems in nature, drive advanced camera lenses in miniaturized imaging systems to enhance the optical performance such as the FOV, resolution, and aberration. Recently, such miniaturized imaging lens systems have been actively exploited for biomedical applications and self‐driving technology . The developments of miniaturized imaging lens have demanded an image detector with high optical performances including a wide FOV and high resolution in a small form factor.…”

“…In addition, ACE provides various optical properties: a wide FOV, fast motion sensitivity, and polarization sensitivity. Thanks to advanced microelectromechanical system (MEMS) fabrication technologies, various ACE structures including planar arrays, curved arrays, and micropolarizer are recently accomplished ( Figure ). Many studies of planar ACE structures have been performed to directly mount the ACE on the CMOS or CCD image sensors.…”

“…This problem was solved with a deformable membrane with photodiodes and the elastomeric microlens arrays was then transformed into a hemispherical shape after bonding with the photodiode arrays. In addition, the compound eye camera is employed for the endoscopic imaging that requires a wide FOV . Since an endoscopy with narrow FOV has a risky potential for missing an important lesion due to a folded shape in the colon of a human, an endoscope capable of observing a wide FOV is required.…”

“…Since 2006, the hemispherical microlens array‐type compound eyes have been widely exploited for man‐made imaging technologies, ranging from fill‐factor enhancement of CMOS and CCD imaging sensors to a compact optical system, along with remarkable advances of micro‐ and nanofabrication techniques . Insect compound eyes have versatile photonic properties, for example, wide field‐of‐view (FOV), negligible distortion, low aberration, and deep depth‐of‐field as well as highly efficient light focusing and guiding, largely exploited for optical color sensing, advanced medical imaging, virtual reality, 3D imaging, light field cameras, robotic systems, security including surveillance and collision‐free navigation of terrestrial and aerospace vehicles . Recently, multilevel and hierarchical fabrications incorporating micro‐ and nanostructures have been developed to reproduce distinguished properties observed in nature.…”

Recent technological advancements in laparoscopic surgical instruments

“…Last Feb. 2017, Cogal and Leblebici [16] presented a miniaturized vision system for endoscopic applications. Inspired by an insect eyes, the authors could develop an imaging system from off-the-shelf miniature cameras integrated with a digital circuit to achieve real-time image processing.…”