Bespoke Fractal Sampling Patterns for Discrete Fourier Space via the Kaleidoscope Transform

White, Jacob M.; Crozier, Stuart; Chandra, Shekhar S.

doi:10.1109/lsp.2021.3116510

Cited by 2 publications

(2 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The KT was recently proposed by White et al [10] to explain the fractal nature of Chaotic Sensing in DFT space [11] and formalises the concept of down-sampling and concatenating an image with itself. For example, a ν, σ-KT decomposes an image into ν 2 down-sampled copies where each is scaled by a factor of σ.…”

Section: Kaleidoscope Embedding For Image Tokensmentioning

confidence: 99%

“…We demonstrate that a KD-based ViT provides an inherently global context of an image, which when cascaded with patch and axial ViT ensures both global and local features are processed independently. This ensemble of ViT is an efficient method of achieving multi-scale perception, as the separate features are attended to without affecting resolution and the Kaleidoscope transform (KT) [10] needs only the computation cost of "patchifying" an image.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Transformer Compressed Sensing Via Global Image Tokens

Lorenzana

Engstrom

Chandra

2022

2022 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

Convolutional neural networks (CNN) have demonstrated outstanding Compressed Sensing (CS) performance compared to traditional, hand-crafted methods. However, they are broadly limited in terms of generalisability, inductive bias and difficulty to model long distance relationships. Transformer neural networks (TNN) overcome such issues by implementing an attention mechanism designed to capture dependencies between inputs. However, high-resolution tasks typically require vision Transformers (ViT) to decompose an image into patch-based tokens, limiting inputs to inherently local contexts. We propose a novel image decomposition that naturally embeds images into low-resolution inputs. These Kaleidoscope tokens (KD) provide a mechanism for global attention, at the same computational cost as a patch-based approach. To showcase this development, we replace CNN components in a well-known CS-MRI neural network with TNN blocks and demonstrate the improvements afforded by KD. We also propose an ensemble of image tokens, which enhance overall image quality and reduces model size. Supplementary material is available: https://github.com/uqmarlonbran/TCS.git.

show abstract

Section: Kaleidoscope Embedding For Image Tokensmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%