In an effort to increase spatial and temporal resolution of ultrafast electron diffraction and microscopy, ultra-high brightness photocathodes are actively sought to improve electron beam quality. Beam dynamics codes often approximate the Coulomb interaction with mean-field space charge, which is a good approximation in traditional beams. However, point-to-point Coulomb effects, such as disorder induced heating and the Boersch effect, can not be neglected in cold, dense beams produced by such photocathodes. In this paper, we introduce two new numerical methods to calculate the important effects of the photocathode image charge when using a point-to-point interaction model. Equipped with an accurate model of the image charge, we calculate the effects of point-topoint interactions on two high brightness photoemission beamlines for ultrafast diffraction. The first beamline uses a 200 keV gun, whereas the second uses a 5 MeV gun, each operating in the singleshot diffraction regime with 10 5 electrons/pulse. Assuming a zero photoemission temperature, it is shown that including stochastic Coulomb effects increases the final emittance of these beamlines by over a factor of 2 and decreases the peak transverse phase space density by over a factor of 3 as compared to mean-field simulations. We then introduce a method to compute the energy released by disorder induced heating using the pair correlation function. This disorder induced heating energy was found to scale very near the theoretical result for stationary ultracold plasmas, and it accounts for over half of the emittance growth above mean-field simulations.