In this study, we investigate the electrical properties of ITO/ZrOx/TaN RRAM devices for neuromorphic computing applications. The thickness and material composition of the device are analyzed using transmission electron microscopy. Additionally, the existence of TaON interface layers was confirmed using dispersive X-ray spectroscopy and X-ray photoelectron analysis. The forming process of the ZrOx-based device can be divided into two categories, namely single- and double forming, based on the initial lattice oxygen vacancies. The resistive switching behaviors of the two forming methods are compared in terms of the uniformity properties of endurance and retention. The rationale behind each I–V forming process was determined as follows: in the double-forming method case, an energy band diagram was constructed using F-N tunneling; conversely, in the single-forming method case, the ratio of oxygen vacancies was extracted based on XPS analysis to identify the conditions for filament formation. Subsequently, synaptic simulations for the applications of neuromorphic systems were conducted using a pulse scheme to achieve potentiation and depression with a deep neural network-based pattern recognition system to display the achieved recognition accuracy. Finally, high-order synaptic plasticity (spike-timing-dependent plasticity (STDP)) is emulated based on the Hebbian rule.