We investigate the Cauchy-Szegő projection for quaternionic Siegel upper half space to obtain the pointwise (higher order) regularity estimates for Cauchy-Szegő kernel and prove that the Cauchy-Szegő kernel is non-zero everywhere, which further yields a non-degenerated pointwise lower bound. As applications, we prove the uniform boundedness of Cauchy-Szegő projection on every atom on the quaternionic Heisenberg group, which is used to give an atomic decomposition of regular Hardy space H p on quaternionic Siegel upper half space for 2/3 < p ≤ 1. Moreover, we establish the characterisation of singular values of the commutator of Cauchy-Szegő projection based on the kernel estimates and on the recent new approach by Fan-Lacey-Li. The quaternionic structure (lack of commutativity) is encoded in the symmetry groups of regular functions and the associated partial differential equations.