It has recently been shown that structural conditions on the reaction network, rather than a 'fine-tuning' of system parameters, often suffice to impart 'absolute concentration robustness' (ACR) on a wide class of biologically relevant, deterministically modelled mass-action systems. We show here that fundamentally different conclusions about the long-term behaviour of such systems are reached if the systems are instead modelled with stochastic dynamics and a discrete state space. Specifically, we characterize a large class of models that exhibit convergence to a positive robust equilibrium in the deterministic setting, whereas trajectories of the corresponding stochastic models are necessarily absorbed by a set of states that reside on the boundary of the state space, i.e. the system undergoes an extinction event. If the time to extinction is large relative to the relevant timescales of the system, the process will appear to settle down to a stationary distribution long before the inevitable extinction will occur. This quasi-stationary distribution is considered for two systems taken from the literature, and results consistent with ACR are recovered by showing that the quasi-stationary distribution of the robust species approaches a Poisson distribution.