Towards Neural Charged Particle Tracking in Digital Tracking Calorimeters With Reinforcement Learning

Abstract

We propose a novel technique for reconstructing charged particles in digital tracking calorimeters using reinforcement learning aiming to benefit from the rapid progress and success of neural network architectures without the dependency on simulated or manually-labeled data. Here we optimize by trial-and-error a behavior policy acting as an approximation to the full combinatorial optimization problem, maximizing the physical plausibility of sampled trajectories. In modern processing pipelines used in high energy physics and related applications, tracking plays an essential role allowing to identify and follow charged particle trajectories traversing particle detectors. Due to the high multiplicity of charged particles and their physical interactions, randomly deflecting the particles, the reconstruction is a challenging undertaking, requiring fast, accurate and robust algorithms. Our approach works on graph-structured data, capturing track hypotheses through edge connections between particles in the detector layers. We demonstrate in a comprehensive study on simulated data for a particle detector used for proton computed tomography, the high potential as well as the competitiveness of our approach compared to a heuristic search algorithm and a model trained on ground truth. Finally, we point out limitations of our approach, guiding towards a robust foundation for further development of reinforcement learning based tracking.

Towards Neural Charged Particle Tracking in Digital Tracking Calorimeters with Reinforcement Learning

“…Track reconstruction and filtering: During inference, particle tracks are constructed combining the trained reconstruction networks (PTT and PAT) in all configurations with the linear assignment solver, creating unique assignments of particle hits to tracks 4 . To remove particle tracks produced by secondary particles or involving inelastic nuclear interactions, we apply a track filtering scheme similar to Pettersen et al ( 2020) and Kortus et al (2023) removing implausible tracks after reconstruction based on physical thresholds. In this work, we limit the track filters to an energy deposition threshold (625 keV in the last reconstructed layer), ensuring the existence of a Bragg peak in the reconstructed track.…”

“…Baseline models: As baseline models, providing reference values for quantifying the track reconstruction performance of the proposed architectures, we select both a traditional iterative track follower algorithm (Pettersen et al, 2020) 5 , and a reinforcement learning (RL) based tracking algorithm (Kortus et al, 2023).…”

“…While both approaches provide great tracking performances, they require a separate optimization step (e.g., clustering) to generate final track candidates in a disconnected step, thus lacking an end-to-end differentiable architecture, preventing gradient flow throughout the whole reconstruction process. Recent progress has been made using reinforcement learning (Kortus et al, 2023) for charged particle tracking, allowing to differentiate through the discrete assignment operation using variants of the policy gradient or score function estimator approach maximizing an objective function J(θ), defined as the expected future reward obtained by the agent's policy.…”

“…Further, each edge is parameterized by a positional encoding, similar to Vaswani et al (2017). However, instead of using discrete token positions, we leverage cosine similarities s cos between two hit segments as proposed by Kortus et al (2023). To reduce the complexity of the graph and to minimize the combinatorial explosion of edges in G L , we remove edges with high angles in G H , measured orthogonal to the sensitive layer.…”

“…Results marked with * us a layer-wise reconstruction scheme with the initial transition generated using ground truth. Results for RL-based are taken from(Kortus et al, 2023)(↑: higher is better; ↓ lower is better)…”