CB-HVT Net: A Channel-Boosted Hybrid Vision Transformer Network for Lymphocyte Detection in Histopathological Images

Abstract

Detection of Tumor-Infiltrating Lymphocytes (TILs) has a high prognostic value in cancer diagnosis due to their ability to identify and kill cancer cells. However, this task is non-trivial due to their diverse morphology, overlapping boundaries , and presence of artifacts. Vision Transformers (ViTs) have the ability to capture long-range relationships, but they lack local correlation in the images and require large training datasets. In this work, we propose a Channel Boosted Hybrid Vision Transformer (CB-HVT) to detect lymphocytes in histopathological images. The proposed network constitutes 1) a channel generation module, 2) a channel exploitation module, 3) a channel merging module, 4) a region-aware module, and 5) a detection and segmentation head. The proposed CB-HVT exploits the learning capacity of both CNN and ViT-based architectures to capture lymphocytic diverse morphology. In addition, we developed a feature fusion block to systematically and gradually merge the diverse feature maps to improve the learning capability of the network. The attention mechanism in the fusion block retains the most contributing features . We evaluated the effectiveness of the proposed CB-HVT on two publicly available datasets for lymphocyte detection in histopathological images . The proposed network showed good results as compared to the existing architectures in terms of F-Score (LYSTO: 0.88 and NuClick: 0.82). In addition, the performance of the proposed CB-HVT on an unseen test set reveals its significance as a valuable tool for pathologists for real-time lymphocyte detection.