W e consider the problem of multimodal-implies the combination of these images into a single ity image fusion, with emphasis on planning, naviga-new (fused) image, which is also registered with the tion and control in neurosurgery. Given that images individual images. to be fused must be registered, we ezplore the issue of It is a rare instance in medical imaging that different image registration in neurosurgery. While rigid regis-imaging modalities, even taken with the same scanner, tration is appropriate for registering images that are would he registered right from the start. Thus, some either all pre-op(erative) or post-op, nonrigid registra-form of registration is usually called for. Registration tion is necessary to register, say, pre-op and intra-op techniques for medical imagery (as well as for other images. This is a consequence of the "brain-shift" oc-applications) are generally divided into rigid transforcuring when the skzlll is openedfor surgery and of issue mations (consisting of translations, rotations, and perretraction and resection, e.g., for tumor removal. We haps scalings) and nonrigid transformations. describe our finite-element-based nonrigid registration when the imaged ob and Qsion systems. Our approach t o fusion 2s znspired ject (here, the brain or cortex) does not deform befrom the human visual system; it was initial% dew[-tween the times the images to be registered are taken. oped for Emote sensing. For examde. if we successivelv collect me-ou MRI-T1 Rigid registration is . ,-. and MRI-TZ images of the same patient, even with the same scanner, registration will probably he necessary since the patient is likely to have moved scans. However, rigid registration will generally be sufficient since the same object is beinn imaged. This is

No abstract

Our goal is to fuse multimodality imagery to enhance image-guided neurosurgery. Images that need to be fused must be registered. Registration becomes a challenge when the imaged object deforms between the times the images to be fused are taken. This is the case when "brain-shift" occurs. We begin by describing our strategy for nonrigid registration via finite-element methods. Then, we independently discuss an image fusion strategy based on a model of the human visual system. We illustrate the operation of many components of the registration system and the operation of the fusion system.