The arrangement of atoms within a crystal and information on deviations from the ideal lattice is encoded in the diffraction pattern obtained from an appropriately conducted Bragg coherent diffraction imaging (BCDI) experiment. A foreknowledge of how specific displacements of atoms within the unit cell alter the BCDI diffraction pattern and the subsequent real-space image is often useful for interpretation and can provide valuable insight for materials design. Here we report on an atomistic approach to efficiently simulate BCDI diffraction patterns by factorising and eliminating certain redundancies in the conventional approach. Our method is able to reduce the computation time by several orders of magnitude without compromising the recovered phase information and therefore enables feasible atomistic simulations on nanoscale crystals with arbitrary lattice distortions.