Multimodal Multi-Head Convolutional Attention with Various Kernel Sizes for Medical Image Super-Resolution

Abstract

Super-resolving medical images can help physicians in providing more accurate diagnostics. In many situations, computed tomography (CT) or magnetic resonance imaging (MRI) techniques output several scans (modes) during a single investigation, which can jointly be used (in a multimodal fashion) to further boost the quality of super-resolution results. To this end, we propose a novel multimodal multi-head convolutional attention module to super-resolve CT and MRI scans. Our attention module uses the convolution operation to perform joint spatial-channel attention on multiple concatenated input tensors, where the kernel (receptive field) size controls the reduction rate of the spatial attention and the number of convolutional filters controls the reduction rate of the channel attention, respectively. We introduce multiple attention heads, each head having a distinct receptive field size corresponding to a particular reduction rate for the spatial attention. We integrate our multimodal multi-head convolutional attention (MMHCA) into two deep neural architectures for super-resolution and conduct experiments on three data sets. Our empirical results show the superiority of our attention module over the state-of-the-art attention mechanisms used in super-resolution. Moreover, we conduct an ablation study to assess the impact of the components involved in our attention module, e.g. the number of inputs or the number of heads. CCS CONCEPTS• Applied computing → Health informatics; • Information systems → Data analytics; • Computing methodologies → Neural networks; Unsupervised learning.

Neural-Network-Based Methods for ARPES Data Processing (Review Article)

“…The attention mechanism [54] has been regarded as an advanced technique to capture long-range dependences. So, some additional complications of the neural network give the possibility to overcome the limitations of the kernels to determine and consider long-range dependences between pixels [55][56][57].…”

Gated Multi-Attention Feedback Network for Medical Image Super-Resolution

“…The PSNR is the ratio between the peak value power and noise power [34]. The PSNR is the most commonly used evaluation index to measure the quality of the lossy transformation reconstruction.…”