Direct optimisation of the discovery significance when training neural networks to search for new physics in particle colliders

Elwood, Adam; Krücker, Dirk

doi:10.48550/arxiv.1806.00322

Cited by 9 publications

(14 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, it is important to consider the task of background estimation in the broader context of analysis optimization. A variety of methods have been proposed to directly optimize analysis sensitivity including uncertainty [107][108][109][110]. Background estimation is a key part of analysis design and could be integrated into the ABCD method in order to further optimize the overall discovery potential.…”

Section: Discussionmentioning

confidence: 99%

ABCDisCo: Automating the ABCD Method with Machine Learning

Kasieczka,

Nachman,

Schwartz

et al. 2020

Preprint

View full text Add to dashboard Cite

The ABCD method is one of the most widely used data-driven background estimation techniques in high energy physics. Cuts on two statistically-independent classifiers separate signal and background into four regions, so that background in the signal region can be estimated simply using the other three control regions. Typically, the independent classifiers are chosen "by hand" to be intuitive and physically motivated variables. Here, we explore the possibility of automating the design of one or both of these classifiers using machine learning. We show how to use state-of-the-art decorrelation methods to construct powerful yet independent discriminators. Along the way, we uncover a previously unappreciated aspect of the ABCD method: its accuracy hinges on having low signal contamination in control regions not just overall, but relative to the signal fraction in the signal region. We demonstrate the method with three examples: a simple model consisting of three-dimensional Gaussians; boosted hadronic top jet tagging; and a recasted search for paired dijet resonances. In all cases, automating the ABCD method with machine learning significantly improves performance in terms of ABCD closure, background rejection and signal contamination.

show abstract

Section: Discussionmentioning

confidence: 99%

ABCDisCo: Automating the ABCD Method with Machine Learning

Kasieczka,

Nachman,

Schwartz

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…In [12,13], an evolutionary method for training neural networks called NeuroEvolution of Augmenting Topologies (NEAT) [14] was used in the direct optimization of event selectors. Another approach to directly optimizing neural networks for discovery significance was introduced in [15], using cost functions defined over batches of training events. Our prescription, on the other hand, relies on simply using the output of typical ML-based classifiers in an optimal manner.…”

Section: The Landscape Of Related Methodsmentioning

confidence: 99%

“…Henceforth, p(e i ) and p i will both refer to this estimate of p(e i ). 15 Out of the six statistical distance measures, only DPearχ 2 maximization leads to a categorizer that is independent of the value of the overall signal to background ratio S/B under H b+s . Training using the other five measures is sensitive to S/B; if training is to be done using these measures, w i and p i both need to be appropriately scaled before proceeding further.…”

Section: Preprocessing: Learning P(e)mentioning

confidence: 99%

“…It is important that the following equation approximately holds true for validation MC events sampled from H b+s : 15 For convenience, we are not introducing new notation, like p, to refer to the estimate of p. 16 In our python implementation ThickBrick [23] there is an option to handle this scaling internally. 17 SUSY and UED searches would be examples where there exists an a priori expectation for the signal cross-section, which is governed by the known gauge couplings and BSM partner spins.…”

Section: Preprocessing: Learning P(e)mentioning

confidence: 99%

See 1 more Smart Citation

Optimal event selection and categorization in high energy physics, Part 1: Signal discovery

Matchev¹,

Shyamsundar²

2019

Preprint

View full text Add to dashboard Cite

We provide a prescription to train optimal machine-learning-based event selectors and categorizers that maximize the statistical significance of a potential signal excess in high energy physics (HEP) experiments, as quantified by any of six different performance measures. For analyses where the signal search is performed in the distribution of some event variables, our prescription ensures that only the information complementary to those event variables is used in event selection and categorization. This eliminates a major misalignment with the physics goals of the analysis (maximizing the significance of an excess) that exists in the training of typical ML-based event selectors and categorizers. In addition, this decorrelation of event selectors from the relevant event variables prevents the background distribution from becoming peaked in the signal region as a result of event selection, thereby ameliorating the challenges imposed on signal searches by systematic uncertainties. Our event selectors (categorizers) use the output of machine-learning-based classifiers as input and apply optimal selection cutoffs (categorization thresholds) that depend on the event variables being analyzed, as opposed to flat cutoffs (thresholds). These optimal cutoffs and thresholds are learned iteratively, using a novel approach with connections to Lloyd's k-means clustering algorithm. We provide a public, Python 3 implementation of our prescription called ThickBrick, along with usage examples.

show abstract

“…The scale factors can also include correction factors such as trigger efficiencies and should account for the sample size when only a subset of the generated data is used to compute the loss. A similar approach of building a differentiable metric based on a FOM was taken by Elwood and Krücker [7], with a loss function based on the discovery significance.…”

Section: Punzi-lossmentioning

confidence: 99%

Punzi-loss:

et al. 2022

View full text Add to dashboard Cite

We present the novel implementation of a non-differentiable metric approximation and a corresponding loss-scheduling aimed at the search for new particles of unknown mass in high energy physics experiments. We call the loss-scheduling, based on the minimisation of a figure-of-merit related function typical of particle physics, a Punzi-loss function, and the neural network that utilises this loss function a Punzi-net. We show that the Punzi-net outperforms standard multivariate analysis techniques and generalises well to mass hypotheses for which it was not trained. This is achieved by training a single classifier that provides a coherent and optimal classification of all signal hypotheses over the whole search space. Our result constitutes a complementary approach to fully differentiable analyses in particle physics. We implemented this work using PyTorch and provide users full access to a public repository containing all the codes and a training example.

show abstract

Direct optimisation of the discovery significance when training neural networks to search for new physics in particle colliders

Cited by 9 publications

References 19 publications

ABCDisCo: Automating the ABCD Method with Machine Learning

ABCDisCo: Automating the ABCD Method with Machine Learning

Optimal event selection and categorization in high energy physics, Part 1: Signal discovery

Punzi-loss:

Contact Info

Product

Resources

About