One of the fundamental challenges in supervised learning for multimodal image registration is the lack of ground-truth for voxel-level spatial correspondence. This work describes a method to infer voxel-level transformation from higher-level correspondence information contained in anatomical labels. We argue that such labels are more reliable and practical to obtain for reference sets of image pairs than voxel-level correspondence. Typical anatomical labels of interest may include solid organs, vessels, ducts, structure boundaries and other subject-specific ad hoc landmarks. The proposed end-to-end convolutional neural network approach aims to predict displacement fields to align multiple labelled corresponding structures for individual image pairs during the training, while only unlabelled image pairs are used as the network input for inference. We highlight the versatility of the proposed strategy, for training, utilising diverse types of anatomical labels, which need not to be identifiable over all training image pairs. At inference, the resulting 3D deformable image registration algorithm runs in real-time and is fully-automated without requiring any anatomical labels or initialisation. Several network architecture variants are compared for registering T2-weighted magnetic resonance images and 3D transrectal ultrasound images from prostate cancer patients. A median target registration error of 3.6 mm on landmark centroids and a median Dice of 0.87 on prostate glands are achieved from cross-validation experiments, in which 108 pairs of multimodal images from 76 patients were tested with high-quality anatomical labels.

Spatially aligning medical images from different modalities remains a challenging task, especially for intraoperative applications that require fast and robust algorithms. We propose a weakly-supervised, label-driven formulation for learning 3D voxel correspondence from higher-level label correspondence, thereby bypassing classical intensity-based image similarity measures. During training, a convolutional neural network is optimised by outputting a dense displacement field (DDF) that warps a set of available anatomical labels from the moving image to match their corresponding counterparts in the fixed image. These label pairs, including solid organs, ducts, vessels, point landmarks and other ad hoc structures, are only required at training time and can be spatially aligned by minimising a crossentropy function of the warped moving label and the fixed label. During inference, the trained network takes a new image pair to predict an optimal DDF, resulting in a fullyautomatic, label-free, real-time and deformable registration. For interventional applications where large global transformation prevails, we also propose a neural network architecture to jointly optimise the global-and local displacements. Experiment results are presented based on cross-validating registrations of 111 pairs of T2-weighted magnetic resonance images and 3D transrectal ultrasound images from prostate cancer patients with a total of over 4000 anatomical labels, yielding a median target registration error of 4.2 mm on landmark centroids and a median Dice of 0.88 on prostate glands.Index Terms-multimodal medical image registration, weakly-supervised learning, prostate cancer

A novel technique for imaging based on ultrawideband (UWB) microwave signals is introduced. Specifically, the procedure deals with the Huygens Principle (HP); using HP to forward-propagate the waves removes the need to solve inverse problems and, consequently, no matrix generation/inversion is required. Together with its simplicity, the methodology permits the capture of contrast-the extent to which different media can be discriminated in the final image. Moreover, UWB allows all the information in the frequency domain to be utilized by combining the information from the individual frequencies to construct a consistent image. It follows that the methodology can identify the presence and location of significant scatterers inside a volume. Validation of the technique through both simulations and measurements on cylinder and spheres with inclusions has been performed. Application of the proposed technique to medical imaging is envisaged

We present an initial experimental validation of a microwave tomography (MWT) prototype for brain stroke detection and classification using the distorted Born iterative method, two-step iterative shrinkage thresholding (DBIM-TwIST) algorithm. The validation study consists of first preparing and characterizing gel phantoms which mimic the structure and the dielectric properties of a simplified brain model with a haemorrhagic or ischemic stroke target. Then, we measure the S-parameters of the phantoms in our experimental prototype and process the scattered signals from 0.5 to 2.5 GHz using the DBIM-TwIST algorithm to estimate the dielectric properties of the reconstruction domain. Our results demonstrate that we are able to detect the stroke target in scenarios where the initial guess of the inverse problem is only an approximation of the true experimental phantom. Moreover, the prototype can differentiate between haemorrhagic and ischemic strokes based on the estimation of their dielectric properties.

We describe an adversarial learning approach to constrain convolutional neural network training for image registration, replacing heuristic smoothness measures of displacement fields often used in these tasks. Using minimallyinvasive prostate cancer intervention as an example application, we demonstrate the feasibility of utilizing biomechanical simulations to regularize a weakly-supervised anatomical-label-driven registration network for aligning pre-procedural magnetic resonance (MR) and 3D intra-procedural transrectal ultrasound (TRUS) images. A discriminator network is optimized to distinguish the registration-predicted displacement fields from the motion data simulated by finite element analysis. During training, the registration network simultaneously aims to maximize similarity between anatomical labels that drives image alignment and to minimize an adversarial generator loss that measures divergence between the predicted-and simulated deformation. The end-to-end trained network enables efficient and fully-automated registration that only requires an MR and TRUS image pair as input, without anatomical labels or simulated data during inference. 108 pairs of labelled MR and TRUS images from 76 prostate cancer patients and 71,500 nonlinear finite-element simulations from 143 different patients were used for this study. We show that, with only gland segmentation as training labels, the proposed method can help predict physically plausible deformation without any other smoothness penalty. Based on cross-validation experiments using 834 pairs of independent validation landmarks, the proposed adversarial-regularized registration achieved a target registration error of 6.3 mm that is significantly lower than those from several other regularization methods.