DETR is the first end-to-end object detector using a transformer encoder-decoder architecture and demonstrates competitive performance but low computational efficiency on high resolution feature maps. The subsequent work, Deformable DETR, enhances the efficiency of DETR by replacing dense attention with deformable attention, which achieves 10× faster convergence and improved performance. Deformable DETR uses the multiscale feature to ameliorate performance, however, the number of encoder tokens increases by 20× compared to DETR, and the computation cost of the encoder attention remains a bottleneck. In our preliminary experiment, we observe that the detection performance hardly deteriorates even if only a part of the encoder token is updated. Inspired by this observation, we propose Sparse DETR that selectively updates only the tokens expected to be referenced by the decoder, thus help the model effectively detect objects. In addition, we show that applying an auxiliary detection loss on the selected tokens in the encoder improves the performance while minimizing computational overhead. We validate that Sparse DETR achieves better performance than Deformable DETR even with only 10% encoder tokens on the COCO dataset. Albeit only the encoder tokens are sparsified, the total computation cost decreases by 38% and the frames per second (FPS) increases by 42% compared to Deformable DETR. Code is available at https://github.com/kakaobrain/sparse-detr.

No abstract

e13553 Background: Programmed death-ligand 1 (PD-L1) expression is a predictive marker for immune checkpoint inhibitors (ICI) treatment in various cancer types. The evaluation of PD-L1 expression level by combined positive score (CPS) correlates with immunotherapeutic response in biliary tract, colorectum, liver, pancreas, prostate, and gastric cancers. This study aimed to assess the performance of an artificial intelligence (AI)-powered PD-L1 CPS analyzer on these six cancer types, and to investigate whether the AI assistance could improve concordance among pathologists. Methods: Lunit SCOPE PD-L1 CPS was developed with 1.51 x 106 tumor cells and 8.73 x 105 immune cells from 2,372 PD-L1 stained whole-slide images (WSI) or tissue microarray cores from various cancer and normal tissues. The algorithm consisted of tissue area segmentation and cell detection AI models. The AI models calculated the CPS by detecting tumor cells over the tumor area and immune cells over the tumor and adjacent area. The model performance was validated on 135 PD-L1 stained WSIs including the six cancer types, which were interpreted by three pathologists. The concordant CPS classification (≥1 or <1) by two or more pathologists was considered as the consensus. Each pathologist revisited to evaluate WSIs with AI assistance (including visualization and scoring) if there was a discrepancy between the pathologist and the AI model. Results: Of 135 WSIs, 122 (90.4%) were classified as the same CPS subgroup by all three pathologists. The CPS ≥1 and <1 subgroup included 67 (49.6%) and 68 (50.4%) cases, respectively. The overall percent agreement (OPA) of the AI model to the consensus of pathologists was 84.4%, ranging from 78.3% (liver) to 91.3% (biliary tract). The AI-assisted re-evaluation by three pathologists was performed in 17, 19, and 22 WSIs, respectively. According to the AI-assisted revision, the unanimous agreement level was increased to 92.6% (125 cases). The OPA of the AI model to the consensus of pathologists was also increased to 91.9%, ranging from 82.6% (liver) to 100.0% (pancreas). Conclusions: This study shows that an AI-powered PD-L1 CPS analyzer can evaluate the CPS in the six cancer types analyzed here at a comparable level to pathologists. AI assistance can improve the concordance of pathologists' CPS interpretation.[Table: see text]

Regularization and transfer learning are two popular techniques to enhance generalization on unseen data, which is a fundamental problem of machine learning. Regularization techniques are versatile, as they are task-and architecture-agnostic, but they do not exploit a large amount of data available. Transfer learning methods learn to transfer knowledge from one domain to another, but may not generalize across tasks and architectures, and may introduce new training cost for adapting to the target task. To bridge the gap between the two, we propose a transferable perturbation, MetaPerturb, which is meta-learned to improve generalization performance on unseen data. MetaPerturb is implemented as a set-based lightweight network that is agnostic to the size and the order of the input, which is shared across the layers. Then, we propose a meta-learning framework, to jointly train the perturbation function over heterogeneous tasks in parallel. As MetaPerturb is a set-function trained over diverse distributions across layers and tasks, it can generalize to heterogeneous tasks and architectures. We validate the efficacy and generality of MetaPerturb trained on a specific source domain and architecture, by applying it to the training of diverse neural architectures on heterogeneous target datasets against various regularizers and fine-tuning. The results show that the networks trained with MetaPerturb significantly outperform the baselines on most of the tasks and architectures, with a negligible increase in the parameter size and no hyperparameters to tune. * : Equal contribution Preprint. Under review.

Cell detection is a fundamental task in computational pathology that can be used for extracting high-level medical information from whole-slide images. For accurate cell detection, pathologists often zoom out to understand the tissuelevel structures and zoom in to classify cells based on their morphology and the surrounding context. However, there is a lack of efforts to reflect such behaviors by pathologists in the cell detection models, mainly due to the lack of datasets containing both cell and tissue annotations with overlapping regions. To overcome this limitation, we propose and publicly release OCELOT, a dataset purposely dedicated to the study of cell-tissue relationships for cell detection in histopathology. OCELOT provides overlapping cell and tissue annotations on images acquired from multiple organs. Within this setting, we also propose multi-task learning approaches that benefit from learning both cell and tissue tasks simultaneously. When compared against a model trained only for the cell detection task, our proposed approaches improve cell detection performance on 3 datasets: proposed OCELOT, public TIGER, and internal CARP datasets. On the OCELOT test set in particular, we show up to 6.79 improvement in F1-score. We believe the contributions of this paper, including the release of the OCELOT dataset at https://lunit-io.github.io/research/ publications/ocelot are a crucial starting point toward the important research direction of incorporating celltissue relationships in computation pathology.