Enhancing the signal-to-noise ratio in avalanche photodiodes by utilizing impact ionization gain requires materials exhibiting low excess noise factors. Amorphous selenium (a-Se) as a wide bandgap at ∼2.1 eV, a solid-state avalanche layer, demonstrates single-carrier hole impact ionization gain and manifests ultralow thermal generation rates. A comprehensive study of the history dependent and non-Markovian nature of hot hole transport in a-Se was modeled using a Monte Carlo (MC) random walk of single hole free flights, interrupted by instantaneous phonon, disorder, hole−dipole, and impact-ionization scattering interactions. The hole excess noise factors were simulated for 0.1−15 μm a-Se thin-films as a function of mean avalanche gain. The hole excess noise factors in a-Se decreases with an increase in electric field, impact ionization gain, and device thickness. The history dependent nature of branching of holes is explained using a Gaussian avalanche threshold distance distribution and the dead space distance, which increases determinism in the stochastic impact ionization process. An ultralow non-Markovian excess noise factor of ∼1 was simulated for 100 nm a-Se thin films corresponding to avalanche gains of 1000. Future detector designs can utilize the nonlocal/non-Markovian nature of the hole avalanche in a-Se, to enable a true solid-state photomultiplier with noiseless gain.