In this study, the effect of defects on the acceptor properties of nitrogen-doped ZnO nanoparticles (NPs) was investigated through the fabrication of light-emitting diodes (LEDs). Nitrogen-doped ZnO NPs were synthesized by an arc discharge in-gas evaporation method and post-annealed at 800 °C in an oxygen and nitrogen atmosphere. The annealed ZnO NPs were characterized by X-ray diffraction, scanning electron microscopy, Raman spectroscopy, and photoluminescence spectroscopy. It was found that the annealing of nitrogen-doped ZnO NPs in a nitrogen environment increased the number of zinc vacancies, while annealing in an oxygen environment increased the number of oxygen vacancies due to nitrogen desorption. The output characteristics of LEDs fabricated with oxygen-annealed NPs were degraded, while those with nitrogen-annealed NPs were significantly improved. From these results, the contribution of zinc vacancies to acceptor formation in ZnO NPs was confirmed for the first time in actual pn junction devices.