Bump hunting in latent space

Bortolato, Blaz; Dillon, Barry M.; Kamenik, Jernej F.; Smolkovič, Aleks

doi:10.1103/physrevd.105.115009

Cited by 27 publications

(11 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This approach would eventually help identify the amount of training data ideally required for reasonable SSL performance. In this study, four different sampling strategies were adopted: (a) random, (b) diverse (by selecting diverse samples from the latent space of the encoder output (Bortolato et al., 2022), (c) random‐augmented, and (d) diverse‐augmented. Although the two former training sample sets (a, b) included imbalanced classes, the latter two (c, d) were augmented via over‐sampling for ensuring balanced classes.…”

Section: Methodsmentioning

confidence: 99%

Self‐supervised learning improves classification of agriculturally important insect pests in plants

Kar

Nagasubramanian

Elango

et al. 2023

The Plant Phenome Journal

View full text Add to dashboard Cite

Insect pests cause significant damage to food production, so early detection and efficient mitigation strategies are crucial. There is a continual shift toward machine learning (ML)‐based approaches for automating agricultural pest detection. Although supervised learning has achieved remarkable progress in this regard, it is impeded by the need for significant expert involvement in labeling the data used for model training. This makes real‐world applications tedious and oftentimes infeasible. Recently, self‐supervised learning (SSL) approaches have provided a viable alternative to training ML models with minimal annotations. Here, we present an SSL approach to classify 22 insect pests. The framework was assessed on raw and segmented field‐captured images using three different SSL methods, Nearest Neighbor Contrastive Learning of Visual Representations (NNCLR), Bootstrap Your Own Latent, and Barlow Twins. SSL pre‐training was done on ResNet‐18 and ResNet‐50 models using all three SSL methods on the original RGB images and foreground segmented images. The performance of SSL pre‐training methods was evaluated using linear probing of SSL representations and end‐to‐end fine‐tuning approaches. The SSL‐pre‐trained convolutional neural network models were able to perform annotation‐efficient classification. NNCLR was the best performing SSL method for both linear and full model fine‐tuning. With just 5% annotated images, transfer learning with ImageNet initialization obtained 74% accuracy, whereas NNCLR achieved an improved classification accuracy of 79% for end‐to‐end fine‐tuning. Models created using SSL pre‐training consistently performed better, especially under very low annotation, and were robust to object class imbalances. These approaches help overcome annotation bottlenecks and are resource efficient.

show abstract

Section: Methodsmentioning

confidence: 99%

Self‐supervised learning improves classification of agriculturally important insect pests in plants

Kar

Nagasubramanian

Elango

et al. 2023

The Plant Phenome Journal

View full text Add to dashboard Cite

show abstract

“…For instance, disentangling semantic features of images via latent spaces in variational autoencoders (VAEs) [76] and its variants [77][78][79][80] has been widely studied in modern ML literature. In the context of collider physics, how latent spaces embed information and can be used as effective candidates for anomaly detection and bump hunting has been studied [24,30,81]. The proponents of PFN performed detailed studies showing how the latent space representation forms discernible contours in the (η, ϕ) plane of jet image representations.…”

Section: Pfnmentioning

confidence: 99%

A detailed study of interpretability of deep neural network based top taggers

Khot

Neubauer

Roy

2023

Mach. Learn.: Sci. Technol.

View full text Add to dashboard Cite

Recent developments in the methods of explainable AI (XAI) allow researchers to explore the inner workings of deep neural networks (DNNs), revealing crucial information about input-output relationships and realizing how data connects with machine learning models. In this paper we explore interpretability of DNN models designed to identify jets coming from top quark decay in high energy proton-proton collisions at the Large Hadron Collider (LHC). We review a subset of existing top tagger models and explore different quantitative methods to identify which features play the most important roles in identifying the top jets. We also investigate how and why feature importance varies across different XAI metrics, how correlations among features impact their explainability, and how latent space representations encode information as well as correlate with physically meaningful quantities. Our studies uncover some major pitfalls of existing XAI methods and illustrate how they can be overcome to obtain consistent and meaningful interpretation of these models. We additionally illustrate the activity of hidden layers as Neural Activation Pattern (NAP) diagrams and demonstrate how they can be used to understand how DNNs relay information across the layers and how this understanding can help to make such models significantly simpler by allowing effective model reoptimization and hyperparameter tuning. These studies not only facilitate a methodological approach to interpreting models butalso unveil new insights about what these models learn. Incorporating these observations into augmented model design, we propose the Particle Flow Interaction Network (PFIN) model and demonstrate how interpretability-inspired model augmentation can improve top tagging performance.

show abstract

“…O iso k-nearest-neighbors(kNN)-based O iso [1] Autoencoder(AE)-based [14][15][16][17][18][19][20][21][22][23] Graph [24], classical k-means clustering [25] O clu kNN-based O clu [1], TS [13] t-score [2,12,26,27], SOFIE [28], ANODE [29], Poissonian Mixture Model [30] CWoLa [31][32][33][34], TNT [35], SALAD [36] SULU [37] UCluster [38] Table 1. A short summary of the isolation-based and clustering-based novelty evaluators/algorithms.…”

Section: Jhep10(2022)085mentioning

confidence: 99%

“…[1], where two kNN-based measures, O iso and O clu , were proposed for novelty evaluation in the AE latent space. After that, a set of metrics based on an ordinary AE [14][15][16], a variational AE [17,18,21], an adversarial AE [14,19,20], and a graph AE [22] were suggested as novelty measures. They mainly include reconstruction error in refs.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Detecting new physics as novelty — Complementarity matters

Jiang

Juste

et al. 2022

J. High Energ. Phys.

View full text Add to dashboard Cite

Novelty detection is a task of machine learning that aims at detecting novel events without a prior knowledge. In particular, its techniques can be applied to detect unexpected signals from new phenomena at colliders. In this paper, we develop an analysis scheme that exploits the complementarity, originally studied in ref. [1], between isolation-based and clustering-based novelty evaluators. This approach can significantly improve the performance and overall applicability of novelty detection at colliders, which we demonstrate using a variety of two dimensional Gaussian samples mimicking collider events. As a further proof of principle, we subsequently apply this scheme to the detection of two significantly different signals at the LHC featuring a $$ t\overline{t}\gamma \gamma $$ t t ¯ γγ final state: $$ t\overline{t}h $$ t t ¯ h , giving a narrow resonance in the diphoton mass spectrum, and gravity-mediated supersymmetry, resulting in broad distributions at high transverse momentum. Compared to existing dedicated searches at the LHC, the sensitivities for detecting both signals are found to be encouraging.

show abstract

Bump hunting in latent space

Cited by 27 publications

References 35 publications

Self‐supervised learning improves classification of agriculturally important insect pests in plants

Self‐supervised learning improves classification of agriculturally important insect pests in plants

A detailed study of interpretability of deep neural network based top taggers

Detecting new physics as novelty — Complementarity matters

Contact Info

Product

Resources

About