Neural Architecture Search for Lightweight Non-Local Networks

Zhang

IEEE Trans. Med. Imaging

et al. 2021

Pathological examination is the gold standard for the diagnosis of cancer. Common pathological examinations include hematoxylin-eosin (H&E) staining and immunohistochemistry (IHC). In some cases, it is hard to make accurate diagnoses of cancer by referring only to H&E staining images. Whereas, the IHC examination can further provide enough evidence for the diagnosis process. Hence, the generation of virtual IHC images from H&Estained images will be a good solution for current IHC examination hard accessibility issue, especially for some low-resource regions. However, existing approaches have limitations in microscopic structural preservation and the consistency of pathology properties. In addition, pixel-level paired data is hard available. In our work, we propose a novel adversarial learning method for effective Ki-67stained image generation from corresponding H&E-stained image. Our method takes fully advantage of structural similarity constraint and skip connection to improve structural details preservation; and pathology consistency constraint and pathological representation network are first proposed to enforce the generated and source images hold the same pathological properties in different staining domains. We empirically demonstrate the effectiveness of our approach on two different unpaired histopathological datasets. Extensive experiments indicate the superior performance of our method that surpasses the state-of-theart approaches by a significant margin. In addition, our approach also achieves a stable and good performance on unbalanced datasets, which shows our method has strong robustness. We believe that our method has significant potential in clinical virtual staining and advance the progress of computer-aided multi-staining histology image analysis.

Section: Discussion and Future Workmentioning

confidence: 99%

Unpaired Stain Transfer Using Pathology-Consistent Constrained Generative Adversarial Networks

Zhang

IEEE Trans. Med. Imaging

et al. 2021

“…[27,34,48] adapts the idea of BERT to vision and language tasks and showed improved performance on multiple sub-tasks. [55] bridges attention and non-local operator to capture longrange dependency, which has been used in many computer vision tasks [67,28,5,64]. In our work, we apply attention over a group of images and show its effectiveness for summarizing information in an image group.…”

Section: Related Workmentioning

confidence: 99%

Context-Aware Group Captioning via Self-Attention and Contrastive Features

Tran

Mai

et al. 2020

Preprint

While image captioning has progressed rapidly, existing works focus mainly on describing single images. In this paper, we introduce a new task, context-aware group captioning, which aims to describe a group of target images in the context of another group of related reference images. Context-aware group captioning requires not only summarizing information from both the target and reference image group but also contrasting between them. To solve this problem, we propose a framework combining selfattention mechanism with contrastive feature construction to effectively summarize common information from each image group while capturing discriminative information between them. To build the dataset for this task, we propose to group the images and generate the group captions based on single image captions using scene graphs matching. Our datasets are constructed on top of the public Conceptual Captions dataset and our new Stock Captions dataset. Experiments on the two datasets show the effectiveness of our method on this new task. 1 * This work has been done during the first author's internship at Adobe.1 Related Datasets and code are released at https://lizw14. github.io/project/groupcap.

“…These networks aim to capture global dependencies in images and videos. Note that non-local neural networks are inspired by the classical non-local method in vision tasks [3] and unlike those in Transformers, the selfattentions in non-local networks are usually not equipped with multi-heads and position embedding [37,4,23]. Afterwards, Transformers achieve remarkable success in NLP tasks [12,25] and, therefore, self-attentions that inherits NLP settings (e.g., multi-heads, position encodings, classification token, etc.)…”

Section: Related Workmentioning

confidence: 99%

Visformer: The Vision-friendly Transformer

Chen¹,

Xie²,

Niu³

et al. 2021

Preprint

The past year has witnessed the rapid development of applying the Transformer module to vision problems. While some researchers have demonstrated that Transformerbased models enjoy a favorable ability of fitting data, there are still growing number of evidences showing that these models suffer over-fitting especially when the training data is limited. This paper offers an empirical study by performing step-by-step operations to gradually transit a Transformer-based model to a convolution-based model. The results we obtain during the transition process deliver useful messages for improving visual recognition. Based on these observations, we propose a new architecture named Visformer, which is abbreviated from the 'Vision-friendly Transformer'. With the same computational complexity, Visformer outperforms both the Transformer-based and convolution-based models in terms of ImageNet classification accuracy, and the advantage becomes more significant when the model complexity is lower or the training set is smaller. The code is available at https://github. com/danczs/Visformer.