Jigsaw-ViT: Learning jigsaw puzzles in vision transformer

Chen, Yingyi; Shen, Xipeng; Liu, Yahui; Tao, Qinghua; Suykens, Johan A. K.

doi:10.1016/j.patrec.2022.12.023

Cited by 16 publications

(4 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Test Accuracy (%) CE [76] 79.4 SELFIE [59] 81.8 PLC [76] 83.4 Nested-CE [8] 84.1 Jigsaw-ViT [9] 89.0 SSR* [18] 88.5 SSR-PASS 89.0 Table 4. Test accuracy (%) on Red Mini-Imagenet (CNWL) [29].…”

Section: Methodsmentioning

confidence: 99%

PASS: Peer-Agreement based Sample Selection for training with Noisy Labels

Garg¹,

Nguyen²,

Felix³

et al. 2023

Preprint

View full text Add to dashboard Cite

Noisy labels present a significant challenge in deep learning because models are prone to overfitting. This problem has driven the development of sophisticated techniques to address the issue, with one critical component being the selection of clean and noisy label samples. Selecting noisy label samples is commonly based on the small-loss hypothesis or on feature-based sampling, but we present empirical evidence that shows that both strategies struggle to differentiate between noisy label and hard samples, resulting in relatively large proportions of samples falsely selected as clean. To address this limitation, we propose a novel peer-agreement based sample selection (PASS). An automated thresholding technique is then applied to the agreement score to select clean and noisy label samples. PASS is designed to be easily integrated into existing noisy label robust frameworks, and it involves training a set of classifiers in a round-robin fashion, with peer models used for sample selection. In the experiments, we integrate our PASS with several state-of-the-art (SOTA) models, including InstanceGM, DivideMix, SSR, FaMUS, AugDesc, and C2D, and evaluate their effectiveness on several noisy label benchmark datasets, such as CIFAR-100, CIFAR-N, Animal-10N, Red Mini-Imagenet, Clothing1M, Mini-Webvision, and Imagenet. Our results demonstrate that our new sample selection approach improves the existing SOTA results of algorithms.

show abstract

Section: Methodsmentioning

confidence: 99%

PASS: Peer-Agreement based Sample Selection for training with Noisy Labels

Garg¹,

Nguyen²,

Felix³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…LiDAR and cameras are commonly used sensors in on-road autonomous driving perception systems [25,31,50], providing crucial information about surrounding objects and their semantics. Convolutional Neural Networks (CNNs) have shown exceptional performance in image [8] and point cloud [50] segmentation, and they have become the core of perception systems in on-road scene understanding. Although many of these systems primarily rely on LiDAR for scene understanding [25,35], there has been increasing interest in using cameras for on-road BEV perception.…”

Section: Related Workmentioning

confidence: 99%

TerrainNet: Visual Modeling of Complex Terrain for High-speed, Off-road Navigation

Meng¹,

Lambert²,

Li³

et al. 2023

Preprint

View full text Add to dashboard Cite

Effective use of camera-based vision systems is essential for robust performance in autonomous off-road driving, particularly in the high-speed regime. Despite success in structured, on-road settings, current end-to-end approaches for scene prediction have yet to be successfully adapted for complex outdoor terrain. To this end, we present TerrainNet, a visionbased terrain perception system for semantic and geometric terrain prediction for aggressive, off-road navigation. The approach relies on several key insights and practical considerations for achieving reliable terrain modeling. The network includes a multi-headed output representation to capture fine-and coarsegrained terrain features necessary for estimating traversability. Accurate depth estimation is achieved using self-supervised depth completion with multi-view RGB and stereo inputs. Requirements for real-time performance and fast inference speeds are met using efficient, learned image feature projections. Furthermore, the model is trained on a large-scale, real-world off-road dataset collected across a variety of diverse outdoor environments. We show how TerrainNet can also be used for costmap prediction and provide a detailed framework for integration into a planning module. We demonstrate the performance of TerrainNet through extensive comparison to current state-of-the-art baselines for camera-only scene prediction. Finally, we showcase the effectiveness of integrating TerrainNet within a complete autonomousdriving stack by conducting a real-world vehicle test in a challenging off-road scenario.

show abstract

“…Chen et al [34] propose an interesting approach to puzzle solving -namely, puzzle solving is used as a Vision Transformer (ViT) component for image classification in natural images. Vision Transformers [35] are deep learning architecture that were primarily created for Natural Language Processing, but they have demonstrated the state-of-the-art performance in computer vision and image classification tasks [36].…”

Section: General Puzzle Solving Algorithmsmentioning

confidence: 99%

Computational techniques for virtual reconstruction of fragmented archaeological textiles

Gigilashvili,

Lukesova,

Gulbrandsen

et al. 2023

Herit Sci

View full text Add to dashboard Cite

Archaeological artifacts play important role in understanding the past developments of the humanity. However, the artifacts are often highly fragmented and degraded, with many details and parts missing due to centuries’ long degradation. Archaeologists and conservators attempt to reconstruct the original state of the objects either physically or virtually. This process includes characterizing and matching fragments’ features to identify which ones belong together. However, this process currently requires an extensive and tedious manual labor. Recent development in computational techniques gave rise to computer-assisted ways of virtual reconstruction, where the computer suggests solutions to the puzzle of scattered fragments and supplements or fully replaces manual labor. However, the capabilities of computational techniques remain limited in many aspects. This review summarizes the state-of-the-art computational techniques for puzzle and virtual reconstruction problems in cultural heritage applications, in general – with a particular interest in archaeological textiles. We overview existing computational methods, their applications and limitations. Afterward, based on the current knowledge gaps, we discuss where the field should go next.

show abstract

Jigsaw-ViT: Learning jigsaw puzzles in vision transformer

Cited by 16 publications

References 38 publications

PASS: Peer-Agreement based Sample Selection for training with Noisy Labels

PASS: Peer-Agreement based Sample Selection for training with Noisy Labels

TerrainNet: Visual Modeling of Complex Terrain for High-speed, Off-road Navigation

Computational techniques for virtual reconstruction of fragmented archaeological textiles

Contact Info

Product

Resources

About