Chemical reaction neural networks (CRNNs) established as the state-of-the-art tool for autonomous mechanism discovery. While they encode some fundamental physical laws, mass- and atom conservation are still violated. We enforce atom conservation by adding a dedicated neural network layer which can be interpreted as constraining the model to physically realizable stoichiometries. Using the standard test cases of the original CRNN paper, we show that the resulting atom conserving chemical reaction neural networks improve training stability and speed, offer robustness against noisy and missing data, and require less data overall. As a result, we anticipate increased model reliability and greater utilization of the potential of real-world data sets. We also discuss the potential of the new atom balance layer for other applications in combustion modeling and beyond, such as mechanism reduction and kinetic surrogate models for reactive flow simulations.