Opticks : GPU Optical Photon Simulation for Particle Physics using NVIDIA® OptiX™

Abstract

Abstract.Opticks is an open source project that integrates the NVIDIA OptiX GPU ray tracing engine with Geant4 toolkit based simulations. Massive parallelism brings drastic performance improvements with optical photon simulation speedup expected to exceed 1000 times Geant4 when using workstation GPUs. Optical photon simulation time becomes effectively zero compared to the rest of the simulation. Optical photons from scintillation and Cherenkov processes are allocated, generated and propagated entirely on the GPU, minimizing transfer overheads and allowing CPU memory usage to be restricted to optical photons that hit photomultiplier tubes or other photon detectors. Collecting hits into standard Geant4 hit collections then allows the rest of the simulation chain to proceed unmodified. Optical physics processes of scattering, absorption, scintillator reemission and boundary processes are implemented in CUDA OptiX programs based on the Geant4 implementations. Wavelength dependent material and surface properties as well as inverse cumulative distribution functions for reemission are interleaved into GPU textures providing fast interpolated property lookup or wavelength generation. Geometry is provided to OptiX in the form of CUDA programs that return bounding boxes for each primitive and ray geometry intersection positions. Some critical parts of the geometry such as photomultiplier tubes have been implemented analytically with the remainder being tessellated. OptiX handles the creation and application of a choice of acceleration structures such as boundary volume hierarchies and the transparent use of multiple GPUs. OptiX supports interoperation with OpenGL and CUDA Thrust that has enabled unprecedented visualisations of photon propagations to be developed using OpenGL geometry shaders to provide interactive time scrubbing and CUDA Thrust photon indexing to enable interactive history selection.

Opticks : GPU Optical Photon Simulation for Particle Physics using NVIDIA® OptiXTM

“…• associating photons with their "gensteps" entirely on the GPU • downloading photons that hit sensors to the CPU using the stream compaction technique • sorting photons by step sequence histories, flags for up to 16 steps are stored Gensteps are the parameters of optical photon producing steps of scintillation or Cerenkov processes, including the number of photons and the line segment along which to generate them. For a detailed description see the prior proceedings [10].…”

“…A major recent advance made possible by the general CSG solid intersection described above is the implementation of automated direct geometry translation allowing the Geant4 geometry model in memory to be directly translated and uploaded to the GPU. The former approach to geometry translation described in [10] required the geometry to be exported and imported via GDML and G4DAE files. The Opticks geometry model is based upon the observation that many elements of a detector geometry are repeated demanding the use of instancing for efficient representation.…”

“…At the end of Geant4 event processing gensteps are uploaded to the GPU and an OptiX kernel is launched to generate and propagate the photons. Further details on the CUDA port of the optical photon simulation can be found in reference [10].…”

“…These proceedings focus on recent major changes to the geometry workflow of Opticks and to the validation approach. Prior proceedings [10] provide further details on NVIDIA OptiX, use of GPU textures and the CUDA port of Geant4 photon generation and optical physics.…”

HEP Software Foundation Community White Paper Working Group - Detector Simulation

“…In order to satisfy the physics accuracy and computing performance requirements of future neutrino experiments, approaches based on new computing platforms are under investigation. For example, Opticks [22] is a software package where the optical physics processes are implemented on the GPU. This package is executed on NVIDIA machines to parallelize the tracing of optical photons and yields a drastic time performance improvement with a speedup of the order of a thousand with respect to the original Geant4 package running on CPUs.…”